Amazon Kinesis Data Analytics

Get actionable insights from streaming data with serverless Apache Flink

Amazon Kinesis Data Analytics is the easiest way to transform and analyze streaming data in real time with Apache Flink. Apache Flink is an open source framework and engine for processing data streams. Amazon Kinesis Data Analytics reduces the complexity of building, managing, and integrating Apache Flink applications with other AWS services.

Amazon Kinesis Data Analytics takes care of everything required to run streaming applications continuously, and scales automatically to match the volume and throughput of your incoming data. With Amazon Kinesis Data Analytics, there are no servers to manage, no minimum fee or setup cost, and you only pay for the resources your streaming applications consume.

Benefits

Powerful real-time processing

Amazon Kinesis Data Analytics provides built-in functions to filter, aggregate, and transform streaming data for advanced analytics. It processes streaming data with sub-second latencies, enabling you to analyze and respond to incoming data and events in real time.

No servers to manage

Amazon Kinesis Data Analytics is serverless; there are no servers to manage. It runs your streaming applications without requiring you to provision or manage any infrastructure. Amazon Kinesis Data Analytics automatically scales the infrastructure up and down as required to process incoming data.

Pay only for what you use

With Amazon Kinesis Data Analytics, you only pay for the processing resources that your streaming applications use. There are no minimum fees or upfront commitments.

Easy to use

Amazon Kinesis Data Analytics enables you to easily and quickly build queries and sophisticated streaming applications in three simple steps: setup your streaming data sources, write your queries or streaming applications, and setup your destination for processed data.

Amazon Kinesis Data Analytics includes open source libraries and runtimes based on Apache Flink that enable you to build an application in hours instead of months using your favorite IDE. The extensible libraries include specialized APIs for different use cases, including stateful stream processing, streaming ETL, and real-time analytics. You can use the libraries to integrate with AWS services like Amazon Managed Streaming for Apache Kafka (Amazon MSK), Amazon Kinesis Data Streams, Amazon Kinesis Data Firehose, Amazon Elasticsearch Service, Amazon S3, Amazon DynamoDB, and more.

Use your favorite language

Amazon Kinesis Data Analytics supports building applications in SQL, Java, Scala, and Python. You can use these languages to easily build applications that perform joins, aggregations over time windows, filters, and more. You can extend the open source libraries and include custom libraries from the language of your choice. Using Amazon Kinesis Data Analytics Studio, you can build applications interactively in SQL, Scala, and Python using Apache Zeppelin notebooks. 

Quickly get started using Amazon Kinesis Data Analytics Studio

With Amazon Kinesis Data Analytics Studio, you can interactively query data streams and rapidly develop stream processing applications using an interactive development environment powered by Apache Zeppelin notebooks. With Amazon Kinesis Data Analytics Studio, streaming data processing is powered by Apache Flink. 

Build streaming applications with Apache Beam

Amazon Kinesis Data Analytics supports running streaming applications built through Apache Beam’s Java SDK in a serverless Apache Flink environment. Apache Beam is an open-source, unified model for defining streaming and batch data processing applications that can be executed across multiple execution engines. You can easily build Apache Beam streaming applications in Java and run them on Amazon Kinesis Data Analytics and other execution engines.

How it works

How Amazon Kinesis Data Analytics works

Use cases

Streaming ETL

You can develop streaming extract-transform-load (ETL) applications with Amazon Kinesis Data Analytics built-in operators to transform, aggregate, and filter streaming data. You can easily deliver your data in seconds to Amazon Kinesis Data Streams, Amazon Managed Streaming for Apache Kafka (Amazon MSK), Amazon Elasticsearch Service, Amazon S3, custom integrations, and more using built-in connectors.

Try out Streaming ETL with Apache Flink and Amazon Kinesis Data Analytics using sample code on GitHub.

Real-time analytics

You can interactively query and analyze data streams in real time and continuously produce insights and results for time sensitive use cases such as click stream analytics. 

Check out our real-time analytics solution briefs on log monitoring and web analytics.

Stateful event processing

You can develop applications that process events from one or more data streams and trigger conditional processing and external actions. You can identify patterns like anomaly detection in your data streams using standard SQL and Apache Flink libraries for complex event processing.

Check out how to enrich your data stream using Amazon Kinesis Data Analytics.

Customers

Autodesk case study
Autodesk computes real-time monitoring metrics such as response time and error-rate spikes for monitoring user experience.
Read case study 
Zynga
Fox computes real-time viewer analytics on live video streaming events like the Super Bowl.
Read blog post 
Zynga
Zynga analyzes real-time game events triggered by player actions at scale.
Watch video 
Palringo case study
Palringo increases user engagement for its mobile community gaming app using real-time metrics.
Read case study 
Gunosy blog post
Gunosy processes 500,000+ records per minute for fast, personalized news curating for end users.
Read blog post 
See customer case studies 

Get started with Amazon Kinesis Data Analytics

Sign up for an AWS account
Sign up for an AWS account

Instantly get access to the AWS Free Tier.

Read the getting started guide
Review the getting started guide

Learn how to use Amazon Kinesis Data Analytics in the step-by-step guide for SQL or Apache Flink.

Start building with Amazon Kinesis Data Analytics
Start building streaming applications

Build your streaming application from the Amazon Kinesis Data Analytics console.