AI is Quicker, More Effective Than Humans in Analyzing Heart Scans

A type of artificial intelligence known as advanced machine learning can classify essential views from heart ultrasound tests faster, more accurately and with less data than board-certified echocardiographers, according to a study led by a UC San Francisco researcher. 

In a study appearing online March 21, 2018, in the journal npj Digital Medicine, researchers used more than 180,000 real-world echocardiogram (echo) images to train a computer to assess the most common echocardiogram views, then tested both the computer and skilled human technicians on new samples. They found that the computers accurately assessed echo videos 91.7-97.8 percent of the time, versus 70.2-83.5 percent for their human counterparts.

“These results suggest our approach may be useful in helping echocardiographers improve their accuracy, efficiency and workflow, and also may provide a foundation for better analysis of echocardiographic data,” said senior author Rima Arnaout, MD, UCSF Health cardiologist and assistant professor in the UCSF Division of Cardiology.

Imaging is an important part of medical diagnosis, but interpreting medical images such as echocardiograms is a complex, time-intensive process that typically requires extensive training. An echo consists of numerous video clips, still images and heart recordings measured from more than a dozen different angles, or “views,” several of which may have only subtle differences. Human interpretation limits accurate and standardized analysis, thereby limiting an echo from its full clinical and research potential for precision medicine.

Deep learning is a powerful, multi-layered form of advanced machine learning that already has proven useful with image-based diagnosis in radiology, pathology, dermatology and other fields. However, it has not yet been widely applied to echocardiograms, partly due to the complexity of their multi-view, multi-modality format. 

In the npj Digital Medicine study, Arnaout and her colleagues used 223,787 images from 267 UCSF Medical Center patients aged 20-96, acquired from 2000-2017. The randomly selected, real-world echo images came from multiple device manufacturers and covered various echo indications, technical qualities and patient variables, including sex and weight.

The researchers built a multilayer neural network and used supervised learning to simultaneously classify 15 standard views. They randomly chose 80 percent of the images (180,294) for training, and reserved 20 percent (43,493) for validation and testing. For performance comparison, each board-certified echocardiographer participating in the study was given 1,500 randomly selected images, 100 from each view, drawn from the same test set given to the model. 

Overall, the computer classified images from 12 video views with 97.8 percent accuracy, Arnaout said. Even on single, low-resolution images, accuracy among 15 views was 91.7 percent, compared to 70.2 to 83.5 percent for the echocardiographers. Additional analysis showed the model found recognizable similarities among related views and made classifications using clinically relevant image features.

In addition to these findings, the researchers learned that by removing color and standardizing the sizes and shapes of videos and still images for training, the images could be reduced in file size without losing their accuracy, allowing for less storage space and easier transmission.

“Our model can be expanded to classify additional sub-categories of echocardiographic view, as well as diseases, work that has foundational utility for research, for clinical practice, and for training the next generation of echocardiographers,” said Arnaout, a member of the UCSF Institute for Computational Health Sciences.

Other contributors to the npj Digital Medicine study were lead author Ali Madani, MS, and Mohammad Mofrad, PhD, of UC Berkeley, and Ramy Arnaout, MD, DPhil, of Beth Israel Deaconess Medical Center. Financial support was provided by the National Institutes of Health National Heart, Lung, and Blood Institute (K08HL125945) and the American Heart Association Precision Medicine Institute (15GPSPG23830004 and 17IGMV33870001).

UC San Francisco (UCSF) is a leading university dedicated to promoting health worldwide through advanced biomedical research, graduate-level education in the life sciences and health professions, and excellence in patient care. It includes top ranked graduate schools of dentistry, medicine, nursing and pharmacy; a graduate division with nationally renowned programs in basic, biomedical, transitional and population sciences; and a preeminent biomedical research enterprise. It also includes UCSF Health, which comprises three top-ranked hospitals, UCSF Medical Center and UCSF Benioff Children's Hospitals in San Francisco and Oakland, and other partner and affiliated hospitals and healthcare providers throughout the Bay Area.