Streaming Data Solution for Amazon MSK

What does this AWS Solution do?

The Streaming Data Solution for Amazon MSK provides AWS CloudFormation templates where data flows through producers, streaming storage, consumers, and destinations. To support multiple use cases and business needs, this solution offers four AWS CloudFormation templates. Similar to the Streaming Data Solution for Amazon Kinesis, the templates are configured to apply best practices to monitor functionality using dashboards and alarms, and to secure data.


Automated configuration
Automatically configure the AWS services necessary to easily capture, store, process, and deliver streaming data.
Four template options
Choose from four different AWS CloudFormation template options. Test new service combinations for your production environment and improve existing applications.
Real-time use cases
Capture high-volume application logs, analyze clickstream data, continuously deliver to a data lake, and more.
Customizable source code
Customize the solution’s boilerplate code, and then use the solution’s monitoring capabilities to quickly transition from testing to production.

AWS Solution overview

The diagrams below present the four AWS CloudFormation templates that you can automatically deploy using the solution's implementation guide.
AWS Architecture Blog
Amazon MSK Backup for Archival, Replay, or Analytics
This post covers patterns and solutions that can be used to backup MSK topics to S3, which enables customers to reduce long-term data retention settings in MSK. Some customers store long term-data in MSK for data analytics and machine learning workloads. We share a pattern to simplify this architecture by offloading topics data in S3 and use S3 for analytics/ML.
Read the full blog post 
Build icon
Deploy an AWS Solution yourself

Browse our library of AWS Solutions to get answers to common architectural problems.

Learn more 
Find an APN partner
Find an AWS Partner Solution

Find AWS Partners to help you get started.

Explore icon
Explore Guidance

Find prescriptive architectural diagrams, sample code, and technical content for common use cases.

Learn more