AWS for Industries
Event-driven medical imaging with AWS HealthImaging and Amazon EventBridge
Healthcare providers collectively acquire billions of medical images each year, and each new image is subject to complex clinical and research workflows that are unique to each organization. New medical images are read, modified, and shared by a myriad of software systems and propagated to other downstream applications. This interoperability is made possible with the DICOM standard data model. At petabyte-scale, monitoring the clinical lifecycle of images in a healthcare enterprise is a tall task.
We will show how healthcare providers, researchers and developers can use Amazon Web Services (AWS) HealthImaging and Amazon EventBridge to build scalable and resilient mechanisms for monitoring medical imaging data changes.
Overview of the Amazon EventBridge integration for AWS HealthImaging
AWS HealthImaging is a HIPAA-eligible service that empowers healthcare providers, researchers and their software partners to store, analyze, and share medical images in the cloud at petabyte scale. HealthImaging stores DICOM data in resources called image sets, which are a collection of image frames and DICOM metadata that have been normalized for consistency. Image sets are created by importing DICOM data into HealthImaging from Amazon Simple Storage Service (Amazon S3) using Import Jobs.
Amazon EventBridge can be used to build event-driven applications at scale across AWS and existing systems. HealthImaging can automatically publish events to EventBridge each time an image set undergoes a state change, and customers can monitor these events to build event-driven architectures. For example, events are generated when an image set is imported, or when an image set is updated.
HealthImaging event notifications are delivered to EventBridge in near real-time and developers can write straightforward rules to listen for specific events. Customers can seamlessly integrate these events in loosely-coupled application integration workflows. They can do this by leveraging various EventBridge features, such as event buses, rules, pipes, and API destinations.
In the following sections we demonstrate, step-by-step, how you can get started with capturing HealthImaging events. Specifically, we show how to use HealthImaging event notifications and EventBridge to extract DICOM header information in a data store and populate a relational database as image sets are created. Please refer to the HealthImaging developer guide for a detailed list of related event types and sample event structures.
Set up a HealthImaging event rule in EventBridge
1. After logging into your AWS Management Console, search for the EventBridge service page and navigate to Rules. Proceed to create a new rule in your desired event bus. It is important to note that by default, EventBridge does not retain events if there is no rule in place to process them.
Figure 1: Rules in the EventBridge console
2. Provide a rule name: example-rule-for-healthimaging-events. Select the Rule with an event pattern option. Click Next.
Figure 2: Defining rule details for EventBridge
3. In the Build event pattern step, proceed with the default options and scroll to the bottom to find the Event pattern section. In the AWS service dropdown, select HealthImaging. Then in the Event type dropdown that appears, select All Events. The event pattern (in JSON) should automatically append key-value pair(s) to match your selection(s). Click Next.
Figure 3: The interactive console UI to define a HealthImaging event pattern
4. Choose the target types and targets to route your events to. (Each rule can support up to five targets.) Target types include: EventBridge event bus, EventBridge API destination or AWS service.
For Target types, select AWS service (default). For Select a target, select CloudWatch log group. For Log Group, create a new log group by typing in a new log group name such as: /aws/events/healthimaging-events. Click Next.
Figure 4: Selecting a target for the rule
5. You can optionally add tags. Click Next.
6. On the Review and create page, review and verify that you have correctly defined your rule. Click Create rule to create your rule.
7. Once the rule has been created, you can test the rule by following the HealthImaging documentation for creating a data store, or starting a DICOM import job.
8. Navigate to the CloudWatch page and then to the Log groups section. Click the /aws/events/healthimaging-events log group. You should see that the EventBridge rule has written log streams that contain HealthImaging events.
We have now demonstrated how HealthImaging makes events available through EventBridge. Let’s discuss how you can use the EventBridge integration for HealthImaging to build compelling event-driven solutions.
Solution design considerations when using EventBridge integration for HealthImaging
When incorporating EventBridge integration for HealthImaging, you should keep in mind the following design considerations:
1. When defining an EventBridge rule or schedule, include only HealthImaging event types that are relevant to the use case.
For example, for the use case of building an image set metadata index, include only the event types that lead to image set metadata changes, such as Image Set Copied, Image Set Created, Image Set Updated, and Image Set Deleted. By including only the event types that the use case needs, you can keep the downstream event handling code clean and efficient. This improves maintainability and testability.
2. When defining an EventBridge rule or schedule, select targets based on resiliency and scalability requirements specific to the use case.
If you only need to send email notifications, you should choose an Amazon Simple Notification Service (Amazon SNS) topic as the target. If you do not need a retry on failure, and expect to process a small batch of image sets that can be processed within the maximum execution time (15 minutes), you can choose an AWS Lambda function as the target.
If you need to retry on failure, or expect to process tens of thousands of image sets in parallel, such as for as image set metadata index use case, you should choose Amazon Simple Queue Service (Amazon SQS) queue as an intermediate target. This way, you can use the Amazon SQS queue to trigger hundreds of Lambda function invocations as the eventual target. You can also associate an Amazon SQS dead-letter queue with the Amazon SQS queue for more fine-grain and robust error handling at the image set level.
3. Configure EventBridge rule or schedule targets according to resiliency and scalability requirements specific to the use case.
When selecting an Amazon SQS queue as an intermediate target, add a redrive policy that:
1) Associates an Amazon SQS dead-letter queue with the Amazon SQS queue
2) Specifies the maximum receives, or the number of retries, before moving a failed event to the Amazon SQS dead-letter queue
Follow the AWS Lambda developer guide to configure the Lambda function timeout, Amazon SQS queue visibility timeout, and Amazon SQS dead-letter queue redrive policy (that is, maximum receives). It is important that you give each Lambda function invocation sufficient execution time to handle successfully the total number of transactions.
Specifically, when configuring the Amazon SQS queue trigger for the Lambda function, appropriately set the following parameters:
- Batch size: For a given Amazon SQS queue, the maximum number of messages that can be in a batch. A single Lambda function invocation must be able to process a total of batch size multiplied by maximum receives transactions within the Lambda function timeout. Lambda function timeout must be less than 15 minutes (900 seconds).
- Batch window: For a given Amazon SQS queue, the maximum amount of time in seconds that messages in a batch can be apart from each other. Amazon SQS will create a batch from messages that fall within a batch window of each other, and up to a total number of messages of batch size, and trigger one Lambda function invocation for this batch of messages. For messages that are further apart than what is allowed within the batch window, Amazon SQS will trigger additional Lambda function invocations.
- Maximum concurrency: For a given Amazon SQS queue, this is the maximum number of concurrent Lambda function invocations that Amazon SQS can trigger.
Solution example: EventBridge event-driven image set metadata propagation
Using EventBridge integration for HealthImaging, you can automatically start event-driven application integration workflows in parallel, and propagate medical image and image metadata changes from upstream to downstream applications. As shown in Figure 5, upstream medical image applications (such as radiology PACS) stores and updates medical images and image metadata in a HealthImaging data store. Downstream applications (such as digital pathology image management system (IMS), research artificial intelligence and machine learning (AI/ML) pipeline, and vendor neutral archive (VNA)) subscribe to HealthImaging events in EventBridge through Amazon SQS queues and AWS Lambda functions.
Figure 5: Amazon EventBridge integration for AWS HealthImaging event-driven workflows
With Amazon SQS queues as triggers for integrated Lambda functions, you can improve resiliency by retrying failed-to-process image or image metadata changes. You can also increase scalability by launching Lambda function invocations in parallel. If processing fails after multiple retries, you can capture the failed-to-process changes by placing the failed image IDs and metadata IDs in Amazon SQS dead-letter queues.
As an example of applying the preceding design considerations, we present an EventBridge event-driven image set metadata propagation solution, known as metadata-index
. This is an EventBridge event-driven solution that demonstrates how to process HealthImaging events and propagate metadata changes at scale and with resiliency. You can follow the instructions in the GitHub repository to build, deploy, and test the solution in your AWS account.
Conclusion
We showed several best practices for using EventBridge integration for HealthImaging to facilitate better resiliency and scalability. Amazon EventBridge integration for AWS HealthImaging enables medical imaging application developers to build clinical and research products that improve clinical efficiency and patient outcomes.
Contact an AWS Representative to know how we can help accelerate your business.
Further reading
- Introducing AWS HealthImaging — purpose-built for medical imaging at scale
- Integration of on-premises medical imaging data with AWS HealthImaging
- Improving medical imaging workflows with AWS HealthImaging and SageMaker
- Building a Scalable DICOM Ingestion Pipeline for AWS HealthImaging with CitiusTech