Choosing an AWS application integration service
Taking the first step
Purpose |
Help determine which AWS application integration services are the best fit for your workloads. |
Last updated |
June 25, 2024 |
Covered services |
Introduction
Application integration is a suite of services that enables communication between decoupled components within microservices, distributed systems, and serverless applications. Amazon Web Services (AWS) offers more than half a dozen application integration services to support a diverse set of workloads running in the cloud.
Choosing an integration service that is the best fit for your organization and workloads can become difficult. This decision guide will help you ask the right questions to discover your requirements and provides clear guidance on how to evaluate and choose the right integration services for your workloads.
This 8½-minute clip is from a one hour recording of a presentation by AWS director of enterprise strategy Gregor Hohpe at AWS re:Invent 2022. It provides an overview of available AWS application integration services.
Understand
One of the key benefits of modernization is the ability to shift operational responsibilities, allowing you to free up resources to do more value-add and innovation-led activities.
There is a spectrum of shared responsibility options across different levels of modernization, ranging from hosting your message broker on Amazon Elastic Compute Cloud (Amazon EC2)—where you are managing scaling, security configurations, provisioning, patching, and more—to serverless offerings where all of the underlying infrastructure is managed.

As you start to explore and understand your criteria, environment, and the suite of integration services that AWS offers, we recommend that you review some best practices. These best practices are applicable regardless of which service (or suite of services) you choose.
Understand integration in your environment
It's common that some organizations spend more time than they would like on maintaining
open-source integrations. We recommend that you consider community sources, and/or backing from
enterprises or foundations when making these investments. An investment in these projects is not
just financial, but also an investment in knowledge capital and potentially technical debt, as
these components and associated integrations will typically need updating. For more information,
see the AWS Open Source
blog
Understand your architecture characteristics
The ability to support a wide range of architectures is important. We recommend that you use
the AWS Well-Architected Framework
Use a combination of integration services
If you are using purpose-built services, a combination of services may be the best fit for your use case. The following lists out a few common ways AWS customers are using a combination of services.
-
Routing Amazon EventBridge or Amazon Simple Notification Service (Amazon SNS) events to an Amazon Simple Queue Service (Amazon SQS) queue as a buffer for downstream consumers.
-
Pulling events directly from a stream (Amazon Kinesis Data Streams or Amazon Managed Streaming for Apache Kafka (Amazon MSK)) or a queue (Amazon SQS or Amazon MQ) with EventBridge Pipes and sending events to an EventBridge bus to push out to consumers.
-
Routing EventBridge or Amazon SNS events to a Kinesis Data Streams or Amazon MSK for gathering and viewing analytics.
Define
Once you have a clearer picture of your criteria, environment, strategic direction, and available services (including both deployment hosted and managed modalities), you need to identify your integration requirements. You might already know some of your requirements if you are migrating to an existing integration platform or message broker. However, you need to establish how these requirements would change if you move to a cloud environment, if at all.
Messaging or streaming platforms
These platforms are expected to fulfill a certain business functionality. Use the following example use cases when considering which functionalities you will need.
Example 1:
Consider an insurance company which receives different claims as messages for different claim types (auto, home, or life) with different business rules. It may mean that the message consumer should have the functionality to route claims to a different destination based on header properties in the message.
Example 2:
Consider an airline in which a flight status update needs to notify all connected systems such as baggage or gate operations using a protocol such as Advanced Messaging Queuing Protocol (AMQP). The big question with functional and business use case primitives is what constitutes a best fit messaging platform. We have multiple choices that can determine the suitability of the platform based on the use case.
-
Market adoption: This platform is widely adopted by a huge customer community and a good enough fit for most use cases. It is tried and tested with a vibrant support community for any issues that may be encountered. It is a low-risk decision with enough training available for the development resources.
-
Best fit for use case: These platforms will be tailored for specific industry use cases such as airlines, logistics, or healthcare. They may be the best fit for those use cases with ready-made templates available for adoption. These platforms can be easy to get started but can lack the level of adoption in the market as well as flexibility. Adopting this type of platform may require extensive time and resources for validation and building in house expertise.
-
Modern: These platforms are built with the next generation architecture to address cloud-scale deployments, multi-tenancy, disaster recovery, and serverless type of pricing. Using this type of platform may require some refactoring of workloads for long-term viability. It uses a cloud-native platform and focuses on using the well-architected principles of modern applications.
Example 3:
If the messaging platform is part of a bigger loan processing workflow which needs to be multi-region, the messaging platform also needs to support the same business requirements. If the business needs the ability to recover and rollback to a previous state in rainy day situations, the underlying messaging or streaming platform also needs capability to have some snapshotting or replay capability to recreate the state of the system.
The integration platform you choose should facilitate asynchronous processing of loan applications or act as the store and forward channel for a multi-step media processing workflow. The criticality of the business process would determine the capabilities needed from the messaging or streaming platform.
Consider
When considering a major application integration architecture in the cloud, there are different ways to determine the functional requirements for each of the integration points.
The following is some of the criteria to consider when choosing an application integration service.
Consider moving to the cloud to reduce operational cost by standardizing on managed services that shift the operational burden to AWS. Higher levels of abstraction allow developers and operators to focus on their own unique value-add activities, instead of undifferentiated tasks.
Choose
Now that you know the criteria you will use to evaluate your application integration needs, you are ready to choose which AWS service or services are right for your workloads in your environment.
Service type | When would you use it? | What is it optimized for? | Associated services |
---|---|---|---|
Capacity |
Use when you need to decouple publishers and subscribers and send events to multiple subscribers simultaneously. |
Optimized for asynchronous, loosely coupled communication between publishers and subscribers. Events provide flexibility in message routing and delivery and are well-suited for event-driven architectures where events play a central role in initiating actions or workflows. |
|
Messaging |
Use when you need either pub/sub messaging to broadcast messages to multiple recipients simultaneously, or point-to-point messaging when you need reliable and asynchronous communication between components. |
Optimized for high-throughput, scalable, and reliable asynchronous pub/sub and point-to-point messaging between distributed components. |
|
Streaming |
Use streaming services such as Amazon Kinesis Data Streams and Amazon Managed Streaming for Apache Kafka (MSK) in scenarios that involve handling and processing real-time streaming data. |
Optimized for ingesting, processing, and analyzing large volumes of real-time streaming data for use cases that require real-time analytics, real-time monitoring, data exploration, and other applications that demand the processing of high-velocity data streams. |
|
Workflows |
Use when you need to design, coordinate, and manage workflows or sequences of tasks in an organized and scalable manner. |
Optimized for use cases such as business process management, application orchestration, data pipeline automation, and microservices coordination. Workflows abstract the underlying infrastructure complexity, allowing you to focus on designing and managing your workflows effectively. They are capable of handling dependencies and sequencing, allowing for parallelism and conditional branching while providing fault tolerance, error handling, and retries to ensure reliable workflow execution. |
|
Scheduling |
Use scheduling when you need to automate routine tasks such as data processing, backups, or system health checks. Tasks often need to be run at specific times or intervals, such as every night, hour, or minute. |
Optimized for reliable, time-based tasks with built-in retry logic. Suitable for workflows requiring precise scheduling and integration with various AWS services. |
Use
You should now have a clear understanding of what each AWS application integration service does—and which one might be right for you. To explore how to use and learn more about each of the available AWS application integration services—we have provided a pathway to explore how each of the services work. The following section provides links to in-depth documentation, hands-on tutorials, and resources to get you started.
![]() Getting started with Amazon SNS We show you how to manage topics, subscriptions, and messages using the Amazon SNS console. |
![]() Filter Messages Published to Topics with Amazon SNS and Amazon SQS Learn how to use the message filtering feature of Amazon SNS. |
![]() Introducing message data protection for Amazon SNS This blog post explains what message data protection is and how it works. |
![]() Amazon SNS - Troubleshooting Learn how to view configuration information, monitor processes, and gather diagnostic data about Amazon SNS. |
![]() Build a turn-based game with Amazon DynamoDB and Amazon SNS Learn how to build a multiplayer, turn-based game using Amazon DynamoDB and Amazon Amazon SNS. |
![]() Building event driven architectures Learn how to build on a simple pub/sub implementation using Amazon SNS as our publishing service and Amazon SQS as a subscriber. |
![]() Archiving and replaying messages with Amazon SNS FIFO Learn how to archive and replay messages published to Amazon SNS FIFO, which can be useful in failure recovery and state replication scenarios. |
Explore
Once you have determined which approach best fits your workload for your environment, we recommend that you review these resources to help you begin implementing your approach. You can find service-specific resources in the previous section, and general event-driven architecture resources in the following section.
Architecture diagrams Explore reference architecture diagrams to help you create a highly available, secure, flexible, and cost effective architectures. |
Whitepapers Explore whitepapers to help you get started, and learn best practices around event driven architectures. |
Blogs Explore blogs to help you stay up to date on the latest technologies, and modernize your applications. |