Automating medical imaging workflows has become a pressing need, particularly in emergency radiology, where time-sensitive decisions are required. Traditional protocoling of emergency brain MRI scans relies on radiologists manually reviewing clinical referral texts, a process that is both time-consuming and prone to errors. This process accounts for a significant portion of a radiologist's workload and is subject to variability in interpretation. Errors in protocoling can lead to inappropriate imaging sequences, suboptimal diagnostic accuracy and delays in patient management.
Recent advancements in artificial intelligence, particularly in machine learning (ML) and deep learning (DL), have enabled automated protocoling based on natural language processing (NLP) techniques. AI-driven tools can standardise protocol selection, minimise human errors and reduce radiologists’ cognitive load by automating repetitive decision-making tasks. A recent study evaluated various ML and DL models for predicting MRI protocols and determining the necessity of contrast agents based on clinical referral texts. The findings highlight the potential of AI-driven automation in enhancing efficiency and accuracy in radiology, potentially leading to improved patient outcomes and optimised resource allocation.
Machine Learning and Deep Learning Approaches
The study assessed the performance of three ML algorithms—Naïve Bayes, support vector machine (SVM) and XGBoost—alongside two DL models—BERT and GPT-3.5—for automatic protocoling of emergency brain MRI scans. These models were trained using referral texts collected over three years and tested on a separate dataset. Each model was evaluated on its ability to assign the appropriate MRI protocol and determine whether contrast administration was necessary.
GPT-3.5 emerged as the most effective model, achieving an 84% accuracy in protocol selection and a 91% accuracy in determining the need for contrast agents. BERT also demonstrated high performance, outperforming traditional ML models. Among the ML models, XGBoost performed the best, achieving a 78% accuracy in protocol selection. The study also explored the impact of dataset size and augmentation techniques on model performance, revealing that larger datasets improved accuracy, particularly for DL models. Model performance was influenced by the complexity of clinical cases, with better accuracy observed in straightforward referrals where symptoms clearly aligned with specific imaging protocols. In contrast, ambiguous or nonspecific referrals posed greater challenges for AI models, highlighting the need for continued refinement.
Comparison with Radiologist Performance
To benchmark AI performance, two experienced emergency radiologists manually reviewed the test set and assigned MRI protocols. Their accuracy ranged between 80% and 83% for protocol selection and between 89% and 91% for contrast determination, closely aligning with the best-performing AI models. The findings indicate that AI models can function at a comparable level to trained radiologists, particularly for well-defined cases. The accuracy of AI models in contrast determination was particularly notable, as they achieved results nearly identical to those of human experts.
Recommended Read: Improving Brain Metastasis Detection with AI and Cross-Technique MRI
However, AI models struggled with ambiguous referral texts where human expertise is required to interpret subtle clinical nuances. Cases involving complex differential diagnoses or atypical symptom presentations often resulted in misclassification by the AI models. This suggests that AI-driven automation can be a valuable assistive tool rather than a complete replacement for human decision-making. Additionally, AI models provided rapid results, processing referrals in seconds, whereas radiologists required several hours to manually review the test set. The time efficiency advantage of AI systems could help mitigate the increasing workload faced by radiologists in emergency settings.
Challenges and Future Prospects
Despite promising results, several challenges must be addressed before AI-driven protocoling can be implemented in clinical settings. One key limitation is data privacy, as models like GPT-3.5 require cloud-based processing, raising concerns about patient confidentiality. Healthcare institutions must evaluate whether in-house AI solutions, such as BERT, which do not require external processing, offer a more viable approach for clinical deployment. Additionally, AI models performed better in predicting contrast agent necessity than in selecting the correct protocol, highlighting the complexity of the latter task.
Challenges also stem from the variability in referral texts. Differences in clinician wording, documentation styles and terminology can impact AI performance. While NLP techniques can process textual information efficiently, understanding the full clinical context remains difficult. Further research is needed to refine AI models, improve their interpretability and ensure their seamless integration into clinical workflows. Prospective validation studies and real-world implementation trials will be essential to assess the true impact of AI-driven protocoling on radiology practice. AI models must be tested across diverse healthcare settings to ensure generalisability and consistency in performance.
The study demonstrates that ML and DL models, particularly GPT-3.5 and BERT, hold significant promise for automating emergency brain MRI protocoling. These AI-driven approaches can streamline workflows, reduce cognitive load on radiologists and enhance accuracy in protocol selection. The ability of AI to match or exceed radiologist performance in certain tasks highlights its potential to support clinical decision-making. However, limitations related to data privacy, interpretability and adaptability to varying referral styles must be addressed before widespread clinical adoption.
While challenges related to data privacy and model generalisation remain, AI has the potential to revolutionise radiology by assisting clinicians in making faster and more reliable diagnostic decisions. Even partial automation of protocoling processes could alleviate workflow interruptions for radiologists, allowing them to focus on more complex cases requiring expert interpretation. Continued research and development will be crucial in unlocking the full potential of AI in medical imaging, paving the way for enhanced diagnostic efficiency and improved patient care.
Source: Radiology: Artificial Intelligence
Image Credit: iStock
