AWS for Industries

AI-Assisted Annotation of Medical Images using MONAI Label on AWS

Blog is guest authored by Professor Ken Butcher, Medical Director of the New South Wales Telestroke Service and Andres Diaz-Pinto, Senior Deep Learning engineer at NVIDIA

Annotating medical images accurately and at scale is a key prerequisite for training Artificial Intelligence (AI) models, which are later used for diagnostic support in a clinical production environment. However, image labelling is not only a tedious, labor intensive, and time-consuming task, but also requires expert and domain-specific skills and knowledge. AI-assisted interactive labelling of medical images can reduce the time required for manual labelling and if introduced into the clinical workflow, may provide clinicians with useful data at the time of patient management.

We will demonstrate how to set up and deploy an AI-assisted annotation solution for medical imaging on Amazon Web Services (AWS) based on MONAI Label and 3D Slicer. We will describe how the New South Wales (NSW) Health Telestroke Service team (led by Dr. Ken Butcher) are deploying the solution as part of a project to overcome challenges to post-processing and quantitative lesion labelling of CT brain scans in acute stroke patients.

MONAI Label is one of the key modules within MONAI (Medical Open Network for Artificial Intelligence). MONAI is a framework for building and deploying medical AI that is offered as either open-source or as a part of the NVIDIA AI Enterprise software suite. The software suite for organizations in production provides enterprise support, including security notifications, SLAs and API stability.

MONAI Label is an AI-assisted labelling and learning engine with active learning, which helps researchers and clinicians collaborate, create annotated datasets, and build AI models in a standardized manner. Researchers build and deploy AI-based annotation applications to the MONAI Label Server to make custom AI labelling available to clinicians. MONAI Label integrates with third-party viewers, such as 3D Slicer and OHIF for radiology and Digital Slide Archive (DSA), QuPath, and Computer Vision Annotation Tool (CVAT) for pathology.

In combination with these viewers, MONAI Label reduces the time and effort required for data labelling by up to 75%. This means clinicians only need to expert label 25% of datasets, and for complex 3D datasets, clinicians can label data in one tenth of the time. The underlying AI-based labelling apps are improved further by users submitting new or modified labels when the automatic re-train option is turned on. This solution can be used by imaging researchers, radiologists, pathologists, cardiologists, ophthalmologists, and general practitioners.

Using MONAI Label on AWS, users will benefit from:

  • Availability of a wide range of GPU instances: Access Amazon EC2 GPU instances, when you need them, for how long you need them. This is in contrast to on-premises data centers, which might offer a limited number of GPU instances reserved for medical imaging research.
  • Security and access controls: A wide variety of security tools to encrypt their medical image data in transit and at rest. AWS private subnets and security groups protects data and compute resources through network isolation. Fine granular access controls using Amazon Identify and Access Management (IAM) and Amazon Cognito determine who is authenticated and authorized to use data and the annotation tools.
  • Scalability and high-performance: Users can scale their annotation application up or down based on demand using tools such as AWS Auto Scaling and Elastic Load Balancing.
  • Cost optimization: With AWS’ pay-as-you-go approach for pricing, users do not need to procure dedicated GPU instances for AI training and inference. They can leverage cloud elasticity to scale up and down compute resources as needed with a variety of instance size options. There are different deployment options for cost optimization as well.

Case Study

According to this review article published on Seminars in Neurology, Stroke is the second leading cause of adult death and a major cause of disability worldwide.

The NSW Telestroke Service, a collaboration between the Prince of Wales Hospital, eHealth NSW, the Agency for Clinical Innovation and the Ministry of Health, provides people across NSW in Australia with rapid virtual access to specialists in stroke diagnosis and treatment. The NSW Telestroke Service team, working with AWS  and eHealth NSW’s UCCW Telestroke Project team, is deploying an AI-assisted Telestroke solution on AWS. The aim of this at scale, world first project is to identify the type of stroke (ischaemic or haemorrhagic) and annotate the images by further segmenting lesions related to the specific stroke type.

Ischaemic stroke results from occlusion of a cerebral blood vessel, most often by a blood clot. Haemorrhagic stroke results from rupture of a cerebral blood vessel, causing bleeding in or around the brain. Stroke sub-type is determined with a CT scan of the head.

While CT scans can be acquired quickly and viewed on the scanner console, quantitative assessment requires accurate segmentation, which is generally performed only in the research setting, independent of patient management.

Within five days, the NSW Telestroke Service team, in collaboration with AWS, built the automated labelling workflow solution in order to accelerate image annotations and measurement automation. The following architecture highlights the flow of the DICOM images from the Enterprise Imaging Repository (hosted on AWS) to an Orthanc server deployed in a secure research environment on AWS. The Orthanc server is used to push images to the MONAI Label server where clinicians view the images using 3D Slicer.

Figure 1 – Architecture Diagram of AI Assisted Annotation Solution on AWS

Figure 1 – Architecture Diagram of AI Assisted Annotation Solution on AWS

The 3D Slicer are deployed on an Amazon AppStream 2.0 graphics instance with GPU, which provides users access to 3D Slicer from anywhere for online interactive image annotations. For each annotation task, MONAI Label uses the pre-trained AI model to predict the labels, which are shown as an overlay together with the original medical image in 3D Slicer. MONAI Label server on AWS is deployed on a separate Amazon EC2 instance with an Amazon EC2 Auto-Scaling group. This allows you to scale out a MONAI Label server independently from the image viewer to run large-scale model re-training jobs. You can select from a variety of NVIDIA GPU EC2 instances to run MONAI Label server (for example, Amazon EC2 G5 instances with 24 GiB GPU memory).

Data scientists and AI engineers can build and train their AI models in batch on Amazon EC2 instances based on aggregated annotations, using Python scripts and command line tools. The trained AI model can be deployed to MONAI Label server and shared with clinicians using 3D Slicer afterwards.

If an annotation is not accurate, users can manually correct it and submit the modified annotation to MONAI Label. MONAI Label server can be configured to continuously re-train the model based on model suggestions using active learning strategies. Thus, clinicians can expect continuous improvements to their annotations through iterative feedback. Once the model prediction meets the clinical requirements, the annotation results can be auto-generated in seconds, instead of up to 20 minutes by performing this task manually.

For segmentation tasks in radiology, MONAI Label server utilises either NIfTI (Neuroimaging Informatics Technology Initiative) files on a local file system or DICOM images retrieved using the DICOMweb protocol. You can route DICOM images to a cloud-based DICOM store, like containerized Orthanc on AWS, and from there to a MONAI Label server for annotation. You can follow the instructions in this hands-on workshop tutorial to create the Amazon AppStream 2.0 stack for this deployment.

All of the compute instances for Orthanc, MONAI Label server, and 3D slicer are created within a secure environment, with private subnet and security group for network isolation. The Orthanc server can only be accessed by the MONAI Label server, and the login of AppStream 2.0 instances are controlled by an Amazon Cognito user pool that can be used for identity federation.

As an alternative to a Orthanc server, you can also use a serverless solution to store your DICOM files and serve DICOMweb requests to MONAI Label server. This open source solution, based on native AWS services such as Amazon Simple Storage Service (Amazon S3), AWS Lambda, Amazon CloudFront, is even more cost effective to serve DICOM images to MONAI Label server.

Conclusion

The deployment of MONAI Label server in combination with a medical imaging viewer on AWS provides a comprehensive solution to annotate medical images in a secure environment and at scale. The solution allows clinicians to create annotated datasets quickly to receive accurate automated annotations.

Stroke is a time-critical medical emergency and determining a patient’s stroke type and treatment options quickly is crucial. The solution used by the NSW Telestroke Service helps to achieve this.

We have Amazon CloudFormation templates in the step-by-step hands-on workshop content to automate the deployment of this AI-Assisted Annotation solution on AWS, which can pick up the most recent NVIDIA GPU-Optimized AMI to create an Amazon EC2 instance.

To know what AWS can do for you contact an AWS Representative.

Further Reading

Professor Ken ButcherProfessor Ken Butcher is a stroke neurologist at the University of New South Wales and is Medical Director of the New South Wales Telestroke Service.

Jason Matthews is a Program Director at eHealth NSW with over 30 years of experience in Information Computing Technology and Program & Operational management. Jason is the Program Director of the eHealth NSW Telestroke portfolio, where he works closely with Professor Ken Butcher.

 

 

 

 

 

 

Andres Diaz-PintoAndres Diaz-Pinto is a Senior Deep Learning engineer at NVIDIA. He has more than 5 years of experience in developing Machine Learning, Deep Learning and Computer Vision systems for healthcare applications. He is also a co-creator of MONAI Label and an active contributor to the MONAI Core library.

TAGS: , ,
Gang Fu

Gang Fu

Gang Fu is a Healthcare Solution Architect at AWS. He holds a PhD in Pharmaceutical Science from the University of Mississippi and has over ten years of technology and biomedical research experience. He is passionate about technology and the impact it can make on healthcare.

Alex Lemm

Alex Lemm

Alex Lemm is a Business Development Manager for Medical Imaging at AWS. Alex defines and executes go-to-market strategies with imaging partners and drives solutions development to accelerate AI/ML-based medical imaging research in the cloud. He is passionate about integrating open source ML frameworks with the AWS AI/ML stack.

Qing Liu

Qing Liu

Qing Liu is a Senior Solution Architect at AWS. Qing has more than 10 years of experience working in healthcare IT industry. He is passionate about using healthcare data to drive better insights and improve patient outcomes. In his spare time, he likes to play tennis with his wife and friends.

Yuan Shi

Yuan Shi

Yuan Shi is a senior Data Lab Architect at Amazon Web Service (AWS), helping customers from various verticals to design and develop machine leaning solutions in the cloud. Yuan received her PhD from Singapore-MIT Alliance and has over 10 years of experience in the field of data science and deep learning learning. She is passionate about data, and she loves to extracting insight from data to drive business outcomes.