CAD—Closing the Gap on Sensitivity, Specificity: Hurdling PACS Integration & System Learning Curves

Twitter icon
Facebook icon
LinkedIn icon
e-mail icon
Google icon
The Visia CT Lung System utilizes CAD algorithms to automatically detect actionable lung nodules based on size, shape, density, and anatomical context. Source: MeVis Medical Solutions

Computer-aided detection (CAD) systems have proved their reliability in terms of sensitivity and specificity as a second reader for chest imaging, mammography and breast MRI in standalone studies assessing performance for a selected group. But before you can ask how CAD is impacting diagnosis and proving its value in everyday clinical practice, the interaction (or learning curve) between radiologist and the technology and the balancing act between sensitivity and specificity must be considered.

“There is a huge learning curve no matter what [CAD] system you use because all have the potential to increase the sensitivity with an associated risk to lose specificity,” says Cornelia Schaefer-Prokop, MD, associate professor at the University of Amsterdam Academic Medical Center in the Netherlands and a user of chest CAD systems.

Since the majority of systems on the market today don’t approach a specificity of 90 to 100 percent yet, as a radiologist, the challenge is in distinguishing between the true positives and the false positives—something that is difficult to quantify and introduce into routine work and seems to be dependant on reader experience.

Schaefer-Prokop, who is familiar with chest CAD systems from EDDA Technologies and Riverain Medical, says it is difficult to quantify, in terms of economic outcome, the eventual impact of these systems in clinical routine because it is easier to quantify the positive or negative impact for a selected group of patients—such as 150 patient patients with CT-proven lesions compared to 100 negative patients without lesions, and have six radiologists read in a controlled, study-like environment. Determining whether there will be an increase in sensitivity or specificity in relation to a radiologist’s experience level is easy in this scenario. Schaefer-Prokop maintains that the success of chest CAD systems will depend upon how they fare in routine clinical practice. This means the systems have to run in the background, be integrated with PACS, be easy to handle so as to not interfere or impede workflow and show an image with complete resolution in the same size as the original.

But she doesn’t dispute the potential of CAD. “It is an undoubtable fact that radiologists tend to miss lesions,” she says “and the potential of CAD is to help radiologists find these lesions, increase [diagnostic] confidence or decrease reading time.”

Lung CAD: On the path to PACS integration
True PACS integration is a step that will help increase routine utilization, says Jin Mo Goo, MD, from Seoul National University Hospital, a researcher in thoracic imaging and clinical applications of CAD, who recently presented his latest research on CAD [EDDA Technologies’ IQQA-Chest CAD] at the 2nd World Congress of Thoracic Imaging in Spain.

“For these CAD tools to be more clinically relevant, these applications should be incorporated into PACS,” Mo Goo says. “Because the ultimate goal of nodule evaluation is to determine nodule malignancy, tools to characterize lesions should be developed.”

He notes that CAD application in lung nodule detection on chest CT can enhance performance regardless of experience level, however, lung nodule detection on chest radiography may be dependant upon a reader’s experience because “the rejection of false-positive CAD marks on those images is not as easy as chest CT. In addition, readers need to understand that there could be considerable variability in the measurement results provided by CAD applications, such as nodule volumetry according to the scan and reconstruction parameters of CT and the software itself.”

Radiologists at Waukesha Memorial Hosptial in Waukesha, Wis., are using a CAD system (MeVis Visia CT CAD, MeVis Medical Solutions) as a second reader to aid in detecting lung nodules and calling out potential abnormalities for roughly 500 high-risk patients screened annually with low-dose, high-resolution CT as part of the International Early Lung Cancer Action Program (I-ELCAP). Comprised of a group of 48 institutions in nine countries, I-ELCAP is dedicated to studying the benefits associated with early lung cancer detection by CT screening.

“CAD’s very good at picking up nodules close to pulmonary arteries or veins, which are sometimes difficult to distinguish,” says radiologist