Breakthrough: How AI Uses Videos to Gauge Parkinson's Severity
UCSF researchers use smart phone videos to capture and quantify motor symptoms and provide new clinical insights for personalized treatment.
Despite recent advancements in the treatment of Parkinson’s Disease, it remains a challenge to accurately measure the progression of symptoms like tremors, stiffness and slowing of movement for this neurological disorder without the time and expense of sending patients to a specialized motion lab – until now.
To provide more personalized treatment based on individuals’ disease state and progression, researchers at the UC San Francisco recently developed a video-based machine learning (ML) system that can quantify and validate motor symptom severity in patients with Parkinson’s Disease (PD) right in the clinic.
Their study appeared online in the June 25 issue of NPJ Parkinson’s Disease.
The system uses single-view, seconds-long videos – recorded on devices such as smartphones, tablets and digital cameras – to translate patients’ movements into a comprehensive movement dataset that’s able to predict high- versus low-severity of PD motor symptoms. The researchers designed the system to use a large array of features representing movement characteristics from raw, unedited video of PD patients performing motor tasks. This type of a system was made possible recently, thanks to advancements in machine learning and computer vision that have enabled the development of algorithms that can extract information from video related to movement at key anatomical positions without the need for physical markers like wearable sensors.
“Our framework addresses many of the shortcomings of previous PD research to create a simple yet comprehensive video-based solution to quantifying PD symptoms,” said co-senior study author Reza Abbasi-Asl, PhD, UCSF assistant professor of neurology. “Our approach extracted and identified the most important movement features and used them to train accurate ML models for predicting low- and high-severity motor impairment states.”
Faster and better diagnostics and treatment
The research team used clinical data from 31 Parkinson’s participants who were evaluated at UCSF as part of a multi-day UCSF Parkinson’s Spectrum cohort study. For each patient, they recorded both a full-body video of them walking/gait and a video of a finger tapping task. As part of the study protocol, the standardized video recordings were taken when patients were both on and off dopaminergic medications, which can improve symptoms. The positions of individual joints in each frame were then extracted using Google MediaPipe Solutions computer vision software in collaboration with co-investigator Anupam Pathak, PhD, and Google Research.
The UCSF team then devised a data-driven approach that validated and robustly quantified established clinical movement signs, but also identified new clinical insights, including pinkie finger movements as well as lower limb and axial (skeletal) features of gait that had not previously been evaluated in relation to clinical severity in PD.
“The field of movement disorders is in need of better tools to measure, monitor and track disease signs and symptoms in a straightforward, reliable and objective way,” said study author Jill Ostrem, MD, a neurologist and medical director and division chief of the UCSF Movement Disorders and Neuromodulation Center. “Our study demonstrates this may be possible using simple standardized video recordings.”
The researchers are planning on follow-up studies to further refine their framework, increasing the degree of automation and validating it in larger, representative cohorts. They also plan to extend the framework to incorporate additional motor modalities, such as facial expressions and speech, as well as extending this to home use. They hope to explore the framework’s utility in predicting other outcomes in PD and apply it to other neurological movement disorders such as dystonia and essential tremor.
“Using standard videos combined with interpretable AI techniques, we can assist neurologists in treating patients with Parkinson’s Disease and other neurological movement disorders,” said co-senior study author Simon Little, MBBS, PhD, UCSF associate professor of neurology. “Objective, video-based readouts of Parkinson’s severity could support faster and better diagnostics and treatment in the future.”
About UCSF Health: UCSF Health is recognized worldwide for its innovative patient care, reflecting the latest medical knowledge, advanced technologies and pioneering research. It includes the flagship UCSF Medical Center, which is a top-ranked specialty hospital, as well as UCSF Benioff Children’s Hospitals, with campuses in San Francisco and Oakland, Langley Porter Psychiatric Hospital and Clinics, UCSF Benioff Children’s Physicians and the UCSF Faculty Practice. These hospitals serve as the academic medical center of the University of California, San Francisco, which is world-renowned for its graduate-level health sciences education and biomedical research. UCSF Health has affiliations with hospitals and health organizations throughout the Bay Area. Visit www.ucsfhealth.org. Follow UCSF Health on Facebook or on Twitter.