A commercially available deep learning algorithm can improve radiology residents’ ability to identify abnormal chest x-rays in the emergency department, and may offer a pathway to alleviate rising costs associated with increased imaging use.
Researchers from South Korea tested the algorithm on more than 1,000 consecutive patients from a single emergency department. The algorithm accurately detected abnormal scans and improved the sensitivity of on-call radiology residents, reported Eui Jin Hwang, Seoul National University College of Medicine’s Department of Radiology, and colleagues.
“The interpretation of chest radiographs is a challenging task, requiring experience and expertise,” Hwang et al. explained in the study published Oct. 22 in Radiology.
“If this algorithm is used as a computer-aided diagnosis tool, we believe it has the potential to improve the interpretation of radiographs in the ED by reducing the number of false-negative interpretations.”
For their study, the team trained the algorithm to spot abnormal chest x-rays from a dataset of 1,135 patients (mean age 53 years). The entire cohort received a chest x-ray in the ED between Jan. 1 and March 31, 2017.
Overall, the algorithm notched an area under the ROC curve of 0.95 for detecting relevant abnormalities. It achieved a sensitivity of 88.7% and specificity of 69.6% at a predetermined high-sensitivity cutoff (probability score of 0.16). At the group’s high-specificity cutoff (probability score of 0.46), the algorithm produced a sensitivity of 81.6% and specificity of 90.3%.
Compared to the algorithm, residents interpreted the scans will lower sensitivity (65.6%), but higher specificity (98.1%). After incorporating the deep learning’s metrics however, their sensitivity improved (73.4%) while specificity fell (94.3%).
Despite the tradeoff, the authors wrote that because chest x-rays are used for screening, sensitivity “may be a more important measure of performance” compared to specificity—particularly in the ED.
In a related editorial, Felipe Munera, MD, and Juan C. Infante, MD, both with the University of Miami Miller School of Medicine’s Department of Radiology, argued that deep learning algorithms—like the one in this study—could be useful in situations where 24-hour coverage is unavailable; such technology may even help control costs.
“Although our medical system and society seem willing to absorb the costs associated with increased imaging, the increasing rate of imaging utilization appears to remain practically unchecked,” the pair wrote. “Thus, we look forward to the continued development of in-house and/or commercially available software that integrates seamlessly with the picture archiving and communication system environment, the radiologist interpretation, and reporting workflow.”
Matt joined Chicago’s TriMed team in 2018 covering all areas of health imaging after two years reporting on the hospital field. He holds a bachelor’s in English from UIC, and enjoys a good cup of coffee and an interesting documentary.