How healthcare organizations can improve discharge medication safety

Medications are integral for preventing and curing illness, yet medication errors can also be a cause of harm. For example, in 1993 over 7000 patients died in the United States as a result of a medication error. As more medications become available for use, the simultaneous use of multiple medications by a single patient is more likely. It’s important that everyone involved in a patient’s care, especially the patient, be aware of the medications that a patient should be taking. This becomes difficult when multiple clinicians are prescribing medications to the patient. Therefore, curating the home medication list for accuracy is crucial. This occurs through the process of medication reconciliation, where the aim is to identify the most accurate list of medications that a patient is taking and prevent discrepancies. Quantitatively measuring the performance of medication reconciliation has become easier in the age of electronic health records (EHRs). Determining the patient-facing quality and accuracy of that process remains difficult, as it requires significant time and effort to complete manual chart audits. The rising costs of healthcare and narrow margins for healthcare organizations preclude them from dedicating resources to manually review the accuracy of medication lists. As a result, patients and families are frequently sent home with inaccurate medication lists.

A review of the literature and regulatory standards as they relate to medication reconciliation shows that our current standards are low. The Joint Commission states that “a good faith effort to collect this information is recognized as meeting the intent of the requirement” for medication reconciliation. Yet, although many regulatory bodies, including the Institute for Healthcare Improvement and the Centers for Medicare and Medicaid Services, encourage good medication reconciliation, the measurement and reporting of the process remains largely quantitative. Tools and reports within EHRs can easily measure the frequency of “clicking the button.” Learn how healthcare organizations can take the next step to provide safer management of medications to their patients using Amazon Web Services (AWS).

Seattle Children’s Hospital (SCH) strives to provide “hope, care and cures” for children. At the Pittsburgh-based health system University of Pittsburgh Medical Center (UPMC), the goal is to “deliver the right care, in the right way, at the right time to every patient.” At both organizations, safety and informatics leaders were interested in using advanced cloud technologies to improve patient safety at discharge. They worked with AWS and Pariveda to develop machine learning (ML) models for identifying unintended discrepancies in discharge medication lists for hospitalized patients in an academic medical center setting. These provided Pariveda with temporary access to de-identified data. The models targeted eight different scenarios. The goals of the project included:

1. Leveraging cloud storage and ML to create models to easily and accurately identify different medication errors on a discharge medication list in the EHR.

2. Creating a repeatable and scalable technology package that can be used by other healthcare systems with similar goals.

Solution overview

Figure 1. F1 Medication error ML model training architecture

Solution

To achieve the project goals, clinical informatics clinicians at Seattle Children’s and UPMC collaborated with the AWS Health Academic Medicine team and with Pariveda Solutions, a preferred AWS partner. This required multiple steps, including:

• Creation of AWS environments with appropriate permissions for both customer and Pariveda participants
• Extraction of discharge medication data from the EHR
• Agreement on medication error definitions for the purposes of consistent curation
• Creation of training and validation data sets
• Clinician curation of rule-based indicators in the training data set
• Creation of ML models for eight medication error types

From project kick-off to final report out, the project took just under four months. The key challenges included:

• Obtaining complete and accurate data for discharge medications
• Finding clinician time for data curation
• Correlating error definitions with current clinical practices and policies

Outcomes

Using a de-identified data set of 30,000 individual medications across over 2000 patient discharges at each institution, we created ML models to identify eight error types from the complete medication instruction content. Curation of the 30,000 medications in each training data set took nearly 70 hours of clinician time. The ML models performed better than the standard rule-based indicators for nearly all error types. Furthermore, the AutoML functionality struggled to pull relevant information out of the clinical display line and free text fields, especially for complicated prescriptions.

Table 1. F1 Scores for all error types by institution

Lessons learned

This project supplied valuable information for future engagements at all stages of the project. These included:

• A complete and accurate data set is of utmost importance. Clear communication with a dedicated analyst is essential for making sure that the data is pulled at the right time, for the right patients, and with the right information. This not only improves model performance but also reduces clinician curation time.
• Due to the complexity of the prescription writing process and the clinical content of medication instructions, additional solutions are needed to parse the clinical information. We plan to use natural language processing to identify RxNorm codes and specific medication dosing information to improve the identification of discrete information and improve the performance of pre-curation indicators. If we can reduce the time required to curate training data, then larger data sets can be used to strengthen the models.
• Project management with regular meetings and deliverables keep the project on schedule and make sure that milestones are met.
• There is an opportunity to provide “curation as a service” for customers who have limitations in clinician resources.
• Using more complex models (e.g., Deep Learning) may produce stronger and more accurate models.

Conclusion

ML is a viable and valuable option for identified potential improvements in discharge medication list quality. Complete and accurate data, engaged clinical leaders, dedicated analyst resources, and data science expertise are crucial for success. This achievement at Seattle Children’s and UMPC sets the stage for the creation of a “medication list quality ML package” to be scaled at other healthcare organizations. For more information, contact the AWS Academic Medical Center team.