ICF helps FDA accelerate the drug labeling review process with AWS machine learning

Within the Food and Drug Administration’s Center for Drug Evaluation and Research, the Division of Medication Error Prevention and Analysis (DMEPA) plays a critical role. DMEPA reviews premarket and postmarket drug labeling to minimize the risk of medication errors.

In partnership with FDA’s DMEPA team, Amazon Web Services (AWS) Partner ICF is developing a machine learning (ML) prototype known as the Computerized Labeling Assessment Tool (CLAT). The prototype employs innovative applications of optical character recognition (OCR) technology and the novel use of computer vision techniques that will alleviate bottlenecks in and enhance the efficiency of the drug labeling review process.

Maximizing efficiency without sacrificing accuracy

The FDA’s drug labeling review process is both extensive and time-sensitive. Typically, each DMEPA reviewer performs 25 to 50 premarket drug reviews per year, analyzing key details of the labeling components to validate adherence with federal regulations and standards. The goal of such a detailed review process is to ensure that approved drug products are safe and effective. Misinterpreted labeling can result in a variety of medication errors and lead to serious adverse health events, including death.

The review process, which includes back-and-forth communication between reviewers and drug manufacturers, is manual and highly labor-intensive. To address these issues, the FDA sought a way to develop a cutting-edge technology to expedite, standardize, and streamline processes for FDA reviewers evaluating medication labels for potential misinterpretation. The new CLAT solution delivers these key benefits:

• Increased efficiency – CLAT will streamline the review process, allowing users to complete multiple reviews simultaneously, remove subjectivity from reviewing things like color and sizing, and provide users with a centralized tool to track reviews as nothing of this type currently exists on the market.
• Improved accuracy – Through user feedback mechanisms, the ML models continuously learn and improve, enhancing error detection over time.
• Standardized practices – CLAT promotes consistent review practices by standardizing the process and encouraging the submission of high-resolution images for review and eventual publication in public repositories.

How machine learning can modernize a labor-intensive, time-sensitive process

ICF collaborated with the FDA to design and build out an AWS Well-Architected Framework to support the needs of the CLAT application. CLAT is built on top of several AWS machine learning and other services:

• The project uses Amazon DynamoDB, Amazon API Gateway, AWS Lambda, and Amazon Simple Storage Service (Amazon S3). The use of cloud infrastructure promotes adaptability and scalability as the needs of the FDA change and grow.
• Amazon Elastic Container Service (Amazon ECS) instances allow for efficient installation and running of ML software with open source components such as TensorFlow and Tesseract OCR to create, train, and run ML models for image analysis.

This framework successfully resulted in an image training methodology that is applicable to various object detection tasks within the healthcare field. Through our collaboration, ICF and the FDA have developed a quick way to teach computers to identify healthcare-related items on drug labels, while maintaining data privacy and compliance.

Using sequential transfer learning, the computers were initially trained on unrelated, randomized images. This helped the model learn how to distinguish the important elements of an image—the objects you want it to recognize, such as a graphical symbol of an ear on a bottle of medicine intended for administration in the ear—from the unimportant elements or background. A visual representation of this can be seen in the following Figure 1.

Figure 1. Image of an ear icon being trained on randomized photo images. This is the first step of our transfer learning process when training convolutional neural networks to identify medically relevant symbols.

FDA sees early benefits from new ML tools

The initial results from the CLAT prototype are promising. The innovative tool has the potential to expedite and enhance the process of drug labeling reviews. Notably, unlike manual reviews, CLAT has the ability to perform multiple simultaneous checks at a time across multiple labels. Furthermore, all results are automatically tracked, annotated, and stored in a centralized location for future review. These advancements are a significant step forward in streamlining regulatory processes and ensuring timely and accurate assessments.

While the current focus is on investigating potential pathways to streamline the drug label review process with ML, the FDA also has its eye on the future. The agency is actively exploring how the CLAT ML prototype could be used in other review processes.

This project exemplifies the power of AI to revolutionize healthcare practices. By enhancing efficiency and accuracy in drug labeling reviews, CLAT paves the way for improved patient safety and medication use. Learn more about how ICF is using AWS technology to help clients scale their greatest innovations.