What does this AWS Solution do?

Many Amazon Web Services (AWS) customers use streaming data to gain real-time insight into customer activity and immediate business trends. Streaming data, which is generated continuously from thousands of data sources, includes a wide variety of data such as log files from your mobile or web applications, e-commerce purchases, in-game player activity, information from social networks, financial trading floors, or geospatial services, and telemetry from connected devices. This data can help companies make well-informed decisions and proactively respond to changing business conditions.

Amazon Kinesis, a platform for streaming data on AWS, offers powerful services that make it easier to build data processing applications, load massive volumes of streaming data, and analyze it in real time.

The following section assumes basic knowledge of architecting on the AWS Cloud, streaming data, and data analysis.

AWS Solution overview

AWS offers a solution that automatically launches and configures Amazon Kinesis Streams to load streaming data, Amazon Kinesis Analytics to filter and process that data, and Amazon Kinesis Firehose to deliver the data to various data stores for search, storage, or further analytics. The diagram below presents the Streaming Analytics Pipeline architecture you can deploy in minutes using the solution's implementation guide and accompanying AWS CloudFormation template.

 Click to enlarge

Real-Time Analytics with Amazon Kinesis architecture

This solution uses Amazon Kinesis Streams to load streaming data into Amazon Kinesis Analytics. An Amazon Kinesis Analytics application filters and processes the data, and puts it into Amazon Kinesis Firehose. Then, an Amazon Kinesis Firehose delivery stream delivers the analyzed data to various data stores for search, storage, or further analytics. You can choose to put your analyzed data into an Amazon S3 bucket, an Amazon Redshift cluster, an Amazon Elasticsearch Service domain, or an Amazon Kinesis stream.

If you choose to persist raw data, AWS Lambda decodes the data and puts it into Amazon Kinesis Firehose which delivers it to Amazon S3.

Real-Time Analytics with Amazon Kinesis

Version 1.0
Last updated: 12/2016
Author: AWS

Estimated deployment time: 5 min


Real-Time Analytics with Amazon Kinesis reference implementation

Leverage the reference implementation to automatically provision and configure the AWS services necessary to start consuming and analyzing streaming data in minutes.

Multiple External Destinations

Amazon Kinesis Data Analytics allows users to specify up to three external destinations for analyzed data. By default, the Streaming Analytics Pipeline allows users to specify a single external destination for their analyzed data.

Streaming data format

Amazon Kinesis Data Analytics allows you to specify a schema to classify your streaming data before it executes SQL queries against your input Kinesis stream.
Explore all AWS Solutions

Browse our portfolio of AWS-built solutions to common architectural problems.

Learn more 
Find a Partner

Find AWS certified consulting and technology partners to help you get started.

Learn more 
Start building in the console

Sign-up and start exploring our services.

Get started