AWS Public Sector Blog
Securely running AI algorithms for 100,000 users on private data
This post explores the architectural design and security concepts employed by Radboud University Medical Center Nijmegen (Radboudumc) to build a secure artificial intelligence (AI) runtime environment on Amazon Web Services (AWS). Business leaders dealing with sensitive or regulated data will find this post invaluable because it demonstrates a proven approach to using the power of AI while maintaining strict data privacy and security standards.
With the growing demand for healthcare services, the global economy is projected to need an additional 14 million healthcare workers by 2030 based on a report by the World Health Organization (WHO). To improve healthcare outcomes, organizations are increasingly looking towards the power of AI to automate services and support healthcare workers with smarter tools. Insights from the increasing amount of available data contribute to a high level of care. In the healthcare sector, AI is set to impact efficiency across the value chain, from automating administrative tasks to improving more accurate diagnostics.
Validating AI algorithms’ performance through benchmarking is a critical step before they can be integrated into clinical practice. In the scientific community, this benchmarking process is facilitated through challenges that allow comparison and competition to accelerate the development of cutting-edge algorithms for clinical use cases. Challenge organizers provide test data, and participants submit AI algorithms for evaluation against this data. To create a fair and unbiased evaluation, the test data remains hidden from participants. Thus, a secure environment that enables third-party AI solutions to run on sensitive medical data is needed.
Radboudumc, a renowned university medical center in the Netherlands, built a global, scalable, low-latency, and secure machine learning medical imaging analysis platform on AWS. Their solution, grand-challenge.org, is a robust machine learning (ML) platform designed for the end-to-end development and benchmarking of AI algorithms in biomedical imaging. What started as a small-scale, in-house solution for algorithm benchmarking has evolved into the leading public research platform for AI challenges in the field of biomedical imaging. The migration to the AWS Cloud in 2021 played a pivotal role in facilitating and accelerating this remarkable growth, enabling Radboudumc to scale their operations and meet the increasing demand for secure and compliant AI solutions.
Validating AI for healthcare
Organizations dealing with sensitive data, such as in healthcare, finance, and automotive industries, face the dual challenge of maintaining data privacy and security while fostering collaboration and innovation. University medical centers, in particular, sit at the intersection of clinical application and medical research, requiring the ability to securely access and analyze data in partnership with collaborators. Furthermore, these institutions often need to test newly developed algorithms against their own proprietary datasets within a secure and compliant environment without exposing data publicly.
Today, more than 100,000 researchers, developers, and clinicians from around the world use grand-challenge.org to deploy and validate their algorithms against private datasets. Organizers can use grand-challenge.org to securely upload sensitive data and keep it hidden from participants. Participants can submit their AI models as solutions to challenges without granting challenge organizers access to their intellectual property. Both parties retain control over their data and algorithms, fostering trust and confidentiality throughout the process. Grand-challenge.org is using a combination of AWS services that enable the safety, integrity, and scalability of the platform.
Unlocking the full potential of ML algorithms depends on access to high-quality data, both in terms of diversity and volume. While public datasets can provide a solid foundation, organizations can take advantage of their proprietary or regulated data that holds the key to developing impactful AI solutions. The AWS Data Exchange simplifies the process of finding and using third-party data in the cloud, including datasets made available by Radboudumc through the Registry of Open Data on AWS. Examples include histopathology data from the TIGER challenge and computed tomography (CT) images from STOIC project.
Securing data and isolating untrusted algorithms
Testing third-party developer code on sequestered data demands a robust security infrastructure to safeguard against potential risks, such as data exfiltration or improper use of the resources made available to challenge participants. At grand-challenge.org, this is achieved by running the algorithms with Amazon SageMaker. Using SageMaker for algorithm deployment comes with the benefits that underlying infrastructure is automatically provisioned. Network communication within this environment is limited to data transfer in and out of Amazon Simple Storage Service (Amazon S3), and data is encrypted both at rest and in transit. This reduces the attack surface and mitigates the risk of unauthorized access to the data that the algorithms need to run on. The following diagram shows the architecture.
Figure 1. Running SageMaker within its own virtual private cloud (VPC) in a private subnet with only an Amazon S3 gateway and access to two S3 buckets for algorithm inputs and outputs.
Seamless integration with the rest of the application through an event-driven architecture
Submissions to challenges happen in waves throughout the year, resulting in submission peaks as well as quiet periods with few submissions. To efficiently handle this variability in platform traffic, grand-challenge.org uses the elasticity of the cloud, dynamically scheduling SageMaker jobs and scaling in and out based on demand. This approach reduces costs and minimizes operational complexity.
The backbone of this serverless, scalable infrastructure is a combination of Amazon EventBridge, Amazon Simple Queue Service (Amazon SQS), and Amazon Elastic Container Service (Amazon ECS). EventBridge monitors the state changes of SageMaker jobs, while the SQS queue and ECS workers orchestrate queued messages to scale on-demand. This seamless scalability allows grand-challenge.org to efficiently manage a high volume of concurrent submissions and algorithm invocations, providing a smooth and responsive experience for users without the need for constant manual oversight.
Results and future roadmap
Grand-challenge.org has served more than 27,000 global participants to securely evaluate their algorithms against 45.6 terabytes (TB) of protected medical data. The platform has hosted more than 130 public challenges across 122 different organizing teams, facilitating more than 10,000 algorithm submissions. These submissions are run individually against hidden test data, resulting in 512,000 inference jobs.
Figure 2. Bar graph showing an increase in inference jobs in relation to the challenge deadline. This increase is handled seamlessly by the scalability of the compute resources.
The platform’s architecture uses the elasticity of cloud computing, scaling compute resources up and down on demand to run submissions in parallel, eliminating the need for maintaining dedicated infrastructure. In total, participants have required more than 62,000 hours of compute time for inference alone, equivalent to more than seven years of continuous compute on a single machine (this excludes the compute time required for algorithm training).
Figure 3. Map showing that challenge organizers and participants are distributed globally.
Grand-challenge.org’s secure and scalable solution has benefited 2,010 academic organizations worldwide, attesting to its global scale and impact in accelerating ML research and development in an environment requiring compliance with strict data privacy and security protocols.
AWS provides access to more than 130 Health Insurance Portability and Accountability Act (HIPAA) eligible services as well as numerous certifications for industry-relevant global IT and compliance standards, including support for General Data Protection Regulation (GDPR), Health Information Trust Alliance Common Security Framework (HITRUST CSF), Esquema Nacional de Seguridad (ENS) High, Hébergeur de Données de Santé (HDS), and Cloud Computing Compliance Controls Catalog (C5). As an example, the Fachklinikum Mainschleife, a German hospital, migrated its IT infrastructure to the AWS Cloud, meeting strict data privacy requirements and adhering to the data protection requirements set by the GDPR.
Learn how you can build secure and scalable infrastructure in alignment with the European Health Data Space (EHDS) and discover the power of generative AI in healthcare.