Clinical Context

Many healthcare providers complain about the EHRs they use, but one redeeming quality of these systems is that they employ decision support technology to improve patient safety. In theory, these safety features may help thousands or even millions of patients, but have they achieved their potential? A previous study by Holmgren and colleagues assessed the efficacy of EHRs in preventing adverse drug events. The study results were published in the January 2020 issue of BMJ Quality and Safety.^[1]

Researchers assessed results on a case-based simulation test from 1527 hospitals in the United States from 2009 to 2016. They found EHRs effective in preventing adverse drug events in only 54% of cases in 2009, and this result improved modestly to 61.6% in 2016; however, there was some hopeful news. Hospitals that took the test multiple times demonstrated stronger average improvement vs hospitals repeating the test only once.

The current study by Classen and colleagues uses a larger number of hospitals to report a broader view of the efficacy of EHRs in promoting patient safety across different domains.

Study Synopsis and Perspective

Despite modest improvements in their safety performance over the past decade, the EHR systems used in US hospitals fail to detect up to 1 in 3 potentially harmful drug interactions and other medication errors, according to researchers.

They analyzed 10 years of data from the Health [Information Technology (IT)] safety test, which has been endorsed by the National Quality Forum. The Leapfrog Group, an employer watchdog organization, has provided this test to hospitals since 2009 as part of its annual hospital survey.

David C. Classen, MD, MS, from the division of clinical epidemiology at the University of Utah School of Medicine, Salt Lake City, Utah, and colleagues published their findings in the May 1 issue of JAMA Open Network.^[2]

The study sample included 2314 hospitals that had tested their EHRs in at least one year between 2009 and 2018. In these tests, using simulated patients, clinicians entered medication orders that had either injured or killed patients previously to evaluate how well hospital EHRs could identify orders with the potential for patient harm.

All of the hospital EHRs included computerized physician order entry (CPOE) and clinical decision support (CDS) features to improve medication safety. The order types were divided into basic and advanced categories. In the first category were orders in which adverse drug events could be prevented by using basic CDS (ie, drug-allergy, drug-route, drug-drug, drug-dose for single doses, and therapeutic duplication contraindications). The second category required advanced CDS to trigger alerts about potential adverse drug events (ie, drug-laboratory, drug-dose for daily doses, drug-age, drug-diagnosis, and corollary order contraindications).

The test's primary outcome measure was whether the hospital EHR system correctly generated an alert, a warning, or a soft or hard stop after a clinician entered a test order that could have caused an adverse drug event. The performance scores in the study represent the percentage of cases in which this happened.

During the study period, mean hospital scores on the overall test, which included both basic and advanced CDS, increased from 53.9%±18.3% in 2009 to 65.6%±15.4% in 2018. Mean scores in the categories using basic CDS increased from 69.8%±20.8% in 2009 to 85.6%±14.9% in 2018. For the categories representing advanced CDS, the average score increased from 29.6%±22.4% in 2009 to 46.1%±21.6% in 2018.

Too Little Improvement

Electronic health records improved in their ability to detect errors, especially in the areas supported by basic CDS. Moreover, the number of institutions taking the test has increased 10-fold during the past decade; however, the progress made to date falls far short of the expectations for safety with electronic ordering systems.

"EHRs are supposed to ensure safe use of medications in hospitals," said Classen, the study's corresponding author and a professor of internal medicine at University of Utah School of Medicine, Salt Lake City, Utah, in a news release.^[3] "[b]ut they're not doing that. In any other industry, this degree of software failure wouldn't be tolerated. You would never get on an airplane, for instance, if an airline could only promise it could get you to your destination safely two-thirds of the time."

Hospitals performed best on drug-allergy interactions, which can be identified with basic CDS. The lowest-performing category was drug-diagnosis contraindications, which can be detected with advanced CDS; yet every major EHR used by the hospitals in the study had at least one hospital that got 100% of the drug-diagnosis contraindications right, Classen told Medscape Medical News.

"So we know that every single leading EHR can do this," he said. "The question is whether it's activated to do so."

According to the study, EHR vendor choice alone explained only 9.9% of the wide variation among hospitals in IT safety performance. Vendor choice plus hospital characteristics represented 14.6% of the variation. Those characteristics included size, whether a hospital belonged to a healthcare system, and whether it was an academic medical center, "[b]ut the hospital characteristics we tracked are crude," Classen noted.

"There are other elements like culture and sophistication of IT operations that we didn't track, and I think they explain the rest of the variation," he explained. "It's how you implement [CPOE] that determines how safe it is."

Given hospitals' concern about protecting elderly patients from the potential harms of polypharmacy, the study authors expressed perplexity over why hospitals did not do more to improve their use of advanced CDS.

Asked to elaborate on this point, Classen said, "Initially [in the study period], it was about inadequacies in the software. But we're now at the point where EHR vendors have capabilities to do all of this. The barrier now is convincing the hospitals it's worth focusing on."

Disappointing Results

Dean Sittig, PhD, a professor at the School of Biomedical Informatics, The University of Texas Health Science Center at Houston, said he found the results of the study dismaying.

"I was disappointed at how little improvement we've made over the years," he told Medscape Medical News.

The study had good coverage across the United States, he noted, with more than 80% of the hospitals had greater than 100 beds. Approximately 80% of the tested hospitals, he noted, used one of the top 3 EHRs. Although these were not named in the study, he surmised they were Epic, Cerner, and Meditech.

Noting that the choice of EHR vendor accounted for only 10% of the variation in performance, he said, "The hospitals are either deciding they don't want that kind of decision support, or maybe they don't know how to turn it on or use it. Either of those is bad for the hospital."

Although hospitals are responsible for properly implementing the CDS features, Sittig said, the results may also show "the technology doesn't work too well. It could be that drug-age, drug-lab, and drug-diagnosis involve more complicated clinical logic. When the logic gets complicated, it's harder to implement. It's harder to write the rules and keep everything straight. It's hard to maintain your decision-support logic."

Classen observed that the major EHRs allow a lot of customization, which may be part of the explanation for the large differences in safety performance.

"Hospitals have so much customization capability that you expect large variation, and that's what we see," he said, "[a]nd in any system where you have this much variation, you know you have safety problems."

Another issue that may cause poor performance, Classen said, is complacency, as clinicians and pharmacists increasingly rely on CPOE systems to catch prescribing errors. Sittig, however, questioned this premise. "Upwards of 90% to 95% of alerts for drug-drug interactions are overridden in most hospitals," he said. "That doesn't imply complacency to me -- it implies they're ignoring the alerts, or someone else is going to catch it."

Hospitals, he added, may be reluctant to turn off CDS alerts because they fear being sued. They would rather have clinicians override them when they get too busy to respond to every warning. Then, if the patient has a bad outcome, it is the physician's responsibility, rather than the hospital's, he pointed out.

This study was supported by grant No. R01HS023696 from the Agency for Healthcare Research and Quality. Classen disclosed grants from the Gordon and Betty Moore Foundation and the Robert Wood Johnson Foundation and serving as an employee of Pascal Metrics outside the submitted work. Sittig disclosed no relevant financial relationships.

Clinical Implications

• A previous study by Holmgren and colleagues found that hospital EHRs were effective in preventing adverse drug events in only 54% of cases in 2009, and this result improved modestly to 61.6% in 2016; however, there was some hopeful news. Hospitals that took the test multiple times demonstrated stronger average improvement vs hospitals repeating the test only once.
• The current study by Classen and colleagues suggested that safety applications within EHRs are far from complete in reducing potential AEs. Drug-allergy contraindication is persistently the highest-performing test.
• Implications for the Healthcare Team: The healthcare team needs to work with technology support to develop meaningful and efficient tools to improve patient safety within EHRs. Communication among the team members of any concerns with CPOE performance and safety concerns should be addressed. The team may also consider developing a committee to address system concerns by clinicians using the CPOE to improve safety.

Earn Credit