AI Algorithm Matches Cardiologists’ Expertise, While Explaining Its Decisions

Clinicians rely daily on electrocardiograms (ECG) to detect common cardiovascular conditions, but accurate diagnoses require high levels of expertise. In a new study that reviewed nearly 1 million ECGs from 365,000 adults, an artificial intelligence algorithm exceeded the performance of a widely available commercial system in nearly all examined diagnoses, while also matching the performance of expert cardiologists and, importantly, providing an explanation for its results.  

Senior author Geoffrey H. Tison, MD, MPH, a cardiologist with the UCSF Division of Cardiology and the Bakar Computational Health Sciences Institute at UCSF, describes key aspects of the “explainable AI” study results, which appear in JAMA Cardiology.  

What are the most important findings of your study? 

Electrocardiograms (ECG) are the most common cardiovascular diagnostic procedure, used in hospitals, clinics and even increasingly on smartphones and smartwatches. They provide critical information about various aspects of heart function, such as rhythm, electrical conduction and structure. 

We developed an AI algorithm, called a “convolutional neural network,” that outperformed a commonly used, commercial ECG analysis algorithm, and was comparable to expert cardiologists for many diagnoses.  

This algorithm was trained using ECG data that is commonly available in most hospitals/healthcare institutions, which means that this same algorithm can be further improved using hundreds of millions of existing ECG data in hospitals worldwide. 

How did the algorithm do?  

It performed strongly for 38 different diagnoses across five broad diagnostic categories, providing among the broadest validations of such an AI approach for ECG diagnoses.  

In addition, we used a technique that adds “explainability” to the algorithm, allowing it to highlight segments of the ECG that are important for each diagnosis that it makes. This provides doctors with important context about why a diagnosis is made and a more detailed view into what may be going on in the heart, and may increase physician confidence with using AI. 

In this study, when study physicians reviewed the automatically highlighted segments that the algorithm learned on its own, it showed that what the algorithm learned “made sense” in terms of what we understand about those diseases and heart functioning. 

Why is your study important or novel? 

Few studies have developed explainable neural networks for ECGs, and none on such a large dataset with such a wide variety of diagnoses. 

This shows that AI algorithms like neural networks can, and perhaps should, be used to  substantially improve the performance of the standard ECG algorithms currently in everyday use. They also offer the added promise to continue improving themselves or “learning” the more data that becomes available. 

By making AI algorithms more “explainable,” doctors and patients may be more likely to understand and therefore trust the diagnoses the algorithms make. And it makes it possible to learn from the algorithms what new previously unrecognized patterns it identified in the data related to certain diseases. 

Most similar studies used specially annotated or curated data that requires many doctors to provide expert labels to “train” the algorithms. Our approach uses existing, readily available labeled data, which is much more efficient and allows data from nearly any hospital worldwide to contribute to model training “as is” without additional annotation. 

What is the long-term significance of your research, how might it be put to use? 

Our results offer strong support for AI algorithms like neural networks to be incorporated into existing commercial ECG algorithms, since they perform better for many diagnoses, can improve over time and provide additional insights through explainability. 

This same algorithm that we developed can now be trained on hundreds of times more data, to further improve its performance and to expand the number of diagnoses that it can accurately make.  

Future work can use the explainability technique we demonstrated, or other similar approaches, to gain new knowledge from AI algorithms by better understanding the minute patterns in the data that they discover from very large amounts of ECG data. For example, this approach makes it possible to discover that certain aspects of the heart’s electrical conduction or chamber structures are abnormal in certain diseases. 

The AI algorithm was developed by researchers at UCSF and UC Berkeley.