Predictions of heart attacks and deaths based on coronary computed tomography angiography (CCTA) are more accurate when made using an artificial intelligence (AI) algorithm than with the Coronary Artery Disease Reporting and Data System (CAD-RADS) or other risk assessment methods, a new study shows.
For the research, published online in Radiology June 25, Kevin Johnson, MD, of Yale and colleagues explored the hypothesis that machine learning could, in tandem with conventional scoring systems, “find a combination of arterial features that better discriminated patients who did not experience an adverse event from those who did.”
The team devised a machine learning algorithm and trained it to use 64 CCTA image features to differentiate between patients who had and had not died or had a subsequent cardiac event.
Using CAD-RADS and four other risk assessment methods, they collected data on risk factors from CCTA studies of 6,882 patients who underwent the procedure between February 2004 and November 2009. Johnson et al. scored patients’ arteries using CAD-RADS and four other risk-assessment methods, then compared these results with prognostic scores derived from a machine learning algorithm that had analyzed the identical risk factor data. Median follow-up time on all patients was nine years.
The comparisons showed that predictions of all-cause mortality, coronary heart disease deaths and myocardial infarctions to be significantly more accurate when generated by the machine learning algorithm than with CAD-RADS. For example, for coronary heart disease deaths, area under the curve (AUC) was 0.85 using the machine learning algorithm and 0.79 using CAD-RADS. Similarly, for coronary heart disease deaths or myocardial infarction, AUC was 0.85 using the machine learning algorithm and 0.80 with CAD-RADS.
The study also revealed that the use of the machine learning model on CCTA studies resulted in better decision-making about whether patients should be prescribed statins. “When deciding whether to start statins, if the choice is made to tolerate treating 45 patients to be sure to include one patient who will later die of coronary disease, the use of the machine-learning score ensures that 93% of patients with events will be administered the drug,” the authors wrote. “If CAD-RADS is used, only 69% will be treated.”
The researchers conceded several limitations in their study design. The work was a retrospective analysis of data acquired at the time of clinical radiology interpretation, and the reproducibility of the data could not be tested. Additionally, they reported, a large number of feature permutations, models and parameters made “a full exploration of the entire space of modeling possibilities” impractical.
Moreover, the incidence of myocardial infarction was underestimated because patient replies to two follow-up letters was incomplete; limited study resources may have led to misclassification bias. “For CAD-RADS the cutoff between very mild and mild stenoses was set at 29%, not 24%, because of limitations in data collection categories before the new classification,” the researchers added. Finally, the study involved a single center and a group of subjects who were, for the most part, Caucasian as well as from one suburban and small city area.
Despite these limitations, however, Johnson et al. concluded that machine learning “can improve the use of vessel features to discriminate between patients who will have an event and those who will not.”