Amazon Kinesis Data Analytics is the easiest way to analyze streaming data in real time. Using templates and built-in operators, you can quickly and easily build queries and sophisticated real-time applications. Amazon Kinesis Data Analytics sets up the resources to run your applications and scales automatically to handle any volume of incoming data.
You do not need to setup and manage a complex infrastructure for high availability and stateful processing. Amazon Kinesis Data Analytics is serverless and takes care of everything required to continuously run your application. This includes automatically provisioning the infrastructure to continuously process streaming data.
Automatic elasticity with pay-as-you-go pricing
Amazon Kinesis Data Analytics elastically scales applications to keep up with any volume of data in the incoming data stream. You only pay for the resources used to run your streaming applications. You need not worry about provisioning infrastructure or paying for idle capacity.
Sub-second processing latency
Amazon Kinesis Data Analytics delivers sub-second processing latencies so you can generate real-time alerts, dashboards, and actionable insights.
For sophisticated applications using Apache Flink
Amazon Kinesis Data Analytics includes open source libraries such as Apache Flink, Apache Beam, AWS SDK, and AWS service integrations. Apache Flink is an open source framework and engine for building highly available and accurate streaming applications with support for Java and Scala. Apache Beam is an open-source, unified model for defining streaming and batch data processing applications that can be executed across multiple execution engines. The AWS SDKs help take the complexity out of coding for many AWS services by providing APIs in your preferred language and includes the AWS libraries, code samples, and documentation.
Flexible APIs are provided that are specialized for different use cases including stateful event processing, streaming ETL, and real-time analytics. Pre-built operators and analytics capabilities enable you to build an Apache Flink streaming application in hours instead of months. The Amazon Kinesis Data Analytics libraries are extensible, enabling you to perform real-time processing for a wide variety of use cases.
Integration with AWS services
You can setup and integrate a data source or destination with minimal code. You can use the Amazon Kinesis Data Analytics libraries to integrate with Amazon S3, Amazon Managed Streaming for Apache Kafka (Amazon MSK), Amazon Elasticsearch Service, Amazon DynamoDB, Amazon Kinesis Data Streams, Amazon Kinesis Data Firehose, Amazon CloudWatch, and AWS Glue Schema Registry.
Advanced integration capabilities
In addition to the AWS integrations, the Amazon Kinesis Data Analytics libraries include more than ten connectors from Apache Flink and the ability to build custom integrations. With a couple more lines of code, you can modify how each integration behaves with advanced functionality. Also, you can build custom integrations using a set of Apache Flink primitives that enable you to read and write from files, directories, sockets, or other sources that you can access over the Internet.
Compatible with AWS Glue Schema Registry
Amazon Kinesis Data Analytics for Apache Flink is compatible with the AWS Glue Schema Registry, a serverless feature of AWS Glue that enables you to validate and control the evolution of streaming data using registered Apache Avro schemas, at no additional charge. The Schema Registry helps you manage your schemas on Amazon Kinesis Data Analytics for Apache Flink workloads that connect to Apache Kafka, Amazon Managed Streaming for Apache Kafka (MSK), or Amazon Kinesis Data Streams, as either a source or sink. When data streaming applications are integrated with the Schema Registry, you can improve data quality and safeguard against unexpected changes using compatibility checks that govern schema evolution.
Exactly once processing
You can use Apache Flink in Amazon Kinesis Data Analytics to build applications whose processed records affect the results exactly once, referred to as exactly once processing. This means that even in the case of an application disruption, like internal service maintenance or user initiated application update, the service will ensure that all data is processed and there is no duplicate data.
The service stores previous and in-progress computations, or state, in running application storage. This enables you to compare real-time and past results over any time period and provides fast recovery during application disruptions. State is always encrypted and incrementally saved in running application storage.
Durable application backups
You can create and delete durable application backups through a simple API call. You can immediately restore your applications from the latest backup after a disruption, or you can restore your application to an earlier version.
For interactive SQL applications
Support for standard SQL
Amazon Kinesis Data Analytics supports standard ANSI SQL, so all you need is familiarity with SQL.
Integrated input and output
Amazon Kinesis Data Analytics integrates with Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose so that you can readily ingest streaming data. Just point Amazon Kinesis Data Analytics at the input stream and it will automatically read the data, parse it, and make it available for processing. You can emit processed results to other AWS services including Amazon S3, Amazon Redshift, and Amazon Elasticsearch Service through Amazon Kinesis Data Firehose. You can also send output data to Amazon Kinesis Data Streams to build advanced stream processing pipelines.
Interactive SQL editor
You get an interactive editor to build SQL queries using streaming data operations like sliding time-window averages. You can also view streaming results and errors using live data to debug or further refine your script interactively.
Easy-to-use schema editor
Amazon Kinesis Data Analytics provides an easy-to-use schema editor to discover and edit the structure of the input data. The wizard automatically recognizes standard data formats such as JSON and CSV. It infers the structure of the input data to create a baseline schema, which you can further refine using the schema editor.
Pre-built SQL templates
The interactive SQL editor comes bundled with a collection of SQL templates that provide baseline SQL code for the most common types of operations such as aggregation, per-event transformation, and filtering. You simply select the template appropriate for your analytics task and then edit the provided code using the SQL editor to customize it for your specific use case.
Advanced stream processing functions
Amazon Kinesis Data Analytics offers functions optimized for stream processing so that you can easily perform advanced analytics such as anomaly detection and top-K analysis on your streaming data.
Get started with Amazon Kinesis Data Analytics
Visit the Amazon Kinesis Data Analytics pricing page.
Build your streaming application from the Amazon Kinesis Data Analytics console.