Google Translates Doctor’s Orders into Spanish and Chinese with Few Significant Errors

In multicultural areas like San Francisco, doctors are increasingly looking to Google Translate to provide written instructions their patients can take home, so they stand a better chance of following medical advice.

But is Translate trustworthy?

Researchers at UC San Francisco say the answer is yes – with some caveats. After analyzing 100 sets of emergency discharge instructions translated by Google’s new machine learning algorithm, which was rolled out in 2017, the researchers concluded it was 92 percent accurate for Spanish and 81 percent accurate for Chinese.

The researchers found that only a small minority of the inaccurate translations – 2 percent in Spanish and 8 percent in Chinese – had the potential to cause clinically significant harm. These were mostly due to grammar or typographical errors in the original written English instructions, which someone who could read English would have been able to understand correctly. 

“Google Translate is more accurate than a lot of clinicians believe, and I think it’s definitely more useful than not providing anything at all,” said Elaine Khoong, MD, MS, a UCSF primary care research fellow and first author of the study, published Feb. 25, 2019, in JAMA Internal Medicine. “We cautiously support its use.”

The algorithm got into trouble, however, when doctors used colloquial terms, like “skip a meal,” a phrase that Google translated into Chinese as “jump over” a meal. 

A more troubling mistranslation occurred when the doctor told the patient to “hold the kidney medicine,” meaning to stop taking it. The algorithm translated that into Spanish as, “keep the medication” and into Chinese as “keep taking” the medication, mistakes that the research team deemed “life threatening.”

Machine translation errors were also linked to the use of medical jargon, and long, complicated sentences, such as, “Please return to the emergency department for worsening abdominal pain, inability to eat or drink due to vomiting, bloody diarrhea, if you pass out or any other concerning symptom.”

Google’s Chinese translation was, “If you pass, vomiting, bloody diarrhea, unable to eat or drink or any other symptoms; come back to the emergency department for treatment.” 

The researchers deemed this a clinically significant error, although they said it was not life threatening.

“It confused the machine because the sentence structure was so complicated,” Khoong said. “It’s certainly an argument for doctors to check themselves when they are writing instructions. We get used to communicating in these very complicated sentences.”

The researchers said Google Translate is best used in combination with human interpreters, even if they are not in the room, who can interpret the physician’s verbal instructions for the patient, over the phone or via video, while the physician shows the patient the written translation on the computer screen or printout. That way, patients can flag errors or confusing passages as they are listening and reading. 

The team also recommended that clinicians give their patients the English version of the written instructions, so English-speaking family members or friends can compare them to the machine translation. 

Khoong said patients would be worse off with no written translation at all. She described a non-English speaking patient with a neck injury, who left the emergency department with a cervical collar but did not understand the need to put it on. 

The patient later showed up with the collar in one hand, and the English discharge instructions in the other, saying, “I know these are important. Can you tell me what they say?”

Authors: Elaine Khoong, MD, MS; Eric Steinbrook; Cortlyn Brown, MD; and Alicia Fernandez, MD, all of UCSF.

Funding: This study was funded by the National Institute of Diabetes and Digestive and Kidney Diseases (K24DK102057) and a National Research Service Award (T32HP19025). 

Disclosures: None.

UC San Francisco (UCSF) is a leading university dedicated to promoting health worldwide through advanced biomedical research, graduate-level education in the life sciences and health professions, and excellence in patient care. It includes top-ranked graduate schools of dentistry, medicine, nursing and pharmacy; a graduate division with nationally renowned programs in basic, biomedical, translational and population sciences; and a preeminent biomedical research enterprise. It also includes UCSF Health, which comprises three top-ranked hospitals – UCSF Medical Center and UCSF Benioff Children’s Hospitals in San Francisco and Oakland – as well as Langley Porter Psychiatric Hospital and Clinics, UCSF Benioff Children’s Physicians and the UCSF Faculty Practice. UCSF Health has affiliations with hospitals and health organizations throughout the Bay Area. UCSF faculty also provide all physician care at the public Zuckerberg San Francisco General Hospital and Trauma Center, and the SF VA Medical Center. The UCSF Fresno Medical Education Program is a major branch of the University of California, San Francisco’s School of Medicine.