An opened source natural language processing (NLP) tool accurately identified incidental pulmonary nodules from unstructured radiology reports, reported NYU Langone Health researchers in a new study.
The tool—SimpleNLP—identified incidental lung nodules (ILNs) with more than 90% sensitivity and more than 80% specificity when tested on a validation set of images, according to a May 24 study published in the Journal of the American College of Radiology. With larger datasets the technique may provide more precise follow-up recommendations.
“Despite the common discovery of the ILN, assessment of radiology reports for lung nodule incidence and guideline concordance of recommendations is understudied,” wrote Stella K. Kang, MD, MSc, with Langone Health’s Department of Radiology in New York, and colleagues.
In the first part of the study, Kang et al. identified patients who underwent a chest CT during 2014 and 2017, before and after the implementation of a department-wide dictation macro of the Fleischner Society recommendations—evidence-based guidelines for follow-up evaluation of ILNs. The team trained and validated their NLP tool against 950 unstructured chest CT reports that were manually reviewed for ILNs.
In the validation set, the NLP picked out ILNs with a 91.1% sensitivity and 82.2% specificity. Positive and negative predictive values were 59.7% and 97%, respectively. The NLP also enabled “exclusion of nodules that were described with definitively benign features that do not require follow-up.”
For part two of the study, the team compared the report’s agreement with the department-wide Fleischner Society guideline dictation macro before and after it was implemented.
Overall there was no difference in the number of reports with ILNs before (21.6% of reports) and after (22.4% of reports) Fleischner dictation was implemented. The same results applied to reports with ILNs containing follow-up recommendations.
Limitations of the study included some necessary manual review of NLP reports to determine if detected nodules were true cases of ILN. This is due to the imperfect specificity of SimpleNLP.
While the tool did not provide overall specific management recommendations nor those specific to imaging features associated with elevated cancer risk immediately after implementation the researchers believe larger datasets could help achieve that goal.
“Application of the SimpleNLP ILN tool to broader data sets of thoracic imaging reports will facilitate evaluation of potential longitudinal trends in recommendation concordance, identification of nodule or study types susceptible to suboptimal reporting, and study of downstream resource use and health outcomes,” they concluded.
Matt joined Chicago’s TriMed team in 2018 covering all areas of health imaging after two years reporting on the hospital field. He holds a bachelor’s in English from UIC, and enjoys a good cup of coffee and an interesting documentary.