Q: What is a streaming data producer?

A: A producer can be thousands of data sources, each generating streaming data continuously and which, typically, submit records simultaneously and in small sizes (kilobytes). Streaming data includes a wide variety of data such as log files generated by customers using mobile or web applications, ecommerce purchases, in-game player activity, information from social networks, financial trading floors, or geospatial services and telemetry from connected devices or instrumentation in data centers. Such data producers generate data using various ingestion patterns to ensure durable and reliable data capture in real-time.

Q: What is streaming data storage?

A: Streaming data must be durably captured by massively scalable storage that is capable of handling high data volume from data producers. One example of streaming storage is Amazon Kinesis Data Streams, a massively scalable and durable real-time data streaming service that enables you to continuously capture data for processing.  


Q: What is a streaming data consumer?

A: Streaming data storage enables the processing of data in real-time by data consumers. Consumers process data sequentially and incrementally, either on a record-by-record basis, or over sliding time windows. The data is used for a wide variety of analytics including correlations, aggregations, filtering, and sampling. Information derived from such analyses gives companies visibility into many business and customer activities such as service usage (for metering/billing), server activity, website clicks, and geo-location of devices, people, and physical goods. Emerging situations can be addressed promptly.

Q: What is a streaming data destination?

A: Streaming data is continuously processed by a data consumer. Afterwards, the processed data is sent to one or more destinations. These destinations can serve as a final storage destination or can be used to chain together streaming data use cases.

Q: Does this solution come with sample datasets and code?

A: Yes, a sample data generator is included as part of the template using direct ingestion and Amazon Kinesis Data Analytics. You can also use the Amazon Kinesis Data Generator to create additional sample datasets.

Q: Where can I find the source code for this solution?

A: You can visit our GitHub repository to download the templates and scripts for this solution and to share your customizations with others.

Q: Does this solution use the AWS CDK?

A: Yes, this solution’s AWS CloudFormation templates are built using the AWS Cloud Development Kit (AWS CDK). The AWS CDK is an open source software development framework to model and provision your cloud application resources using familiar programming languages. The AWS CDK speeds up your development process by providing you the expressive power of programming languages for defining infrastructure.

Q: How do I customize the solution?

A: This solution exposes the essential configurations for each AWS resource that is set up during deployment. After creation, you can update additional configurations directly in the AWS CloudFormation template or through the AWS CDK.

Q: Can I deploy this solution in any AWS Region?

A: This solution uses Amazon Kinesis Data Analytics, which is not currently available in all AWS Regions. Therefore, you must launch this solution in an AWS Region where this service is available. For the most current service availability by Region, refer to the AWS Regional Services List.

Training and Certification

AWS Training and Certification builds your competence, confidence, and credibility through practical cloud skills that help you innovate and build your future.  Learn more »

Introduction to Amazon Kinesis Analytics 

This is an introductory course on Amazon Kinesis Analytics, which helps you query streaming data or build entire streaming applications using SQL.

Enroll now »

Data Analytics Fundamentals

In this self-paced course, you learn about the process for planning data analysis solutions and the various data analytic processes that are involved.

Enroll now »

AWS Certified Solutions Architect – Associate

This exam validates your ability to effectively demonstrate knowledge of how to architect and deploy secure and robust applications on AWS technologies.

Schedule your exam »

Partner resources

The AWS Partner Network (APN) is focused on helping partners build successful AWS-based businesses to drive superb solutions and customer experiences. APN Partners are focused on customer success, helping you take full advantage of all the business benefits that AWS has to offer. With their deep expertise on AWS, APN Partners are uniquely positioned to help your company at any stage of your Cloud Adoption Journey and to help you solve some of your most complex problems.

Need more resources to get started with AWS?

Visit the Getting Started Resource Center to find tutorials, projects and videos to get started with AWS.

Learn more »