Amazon Kinesis Data Analytics is the easiest way to analyze streaming data in real time. Using templates and built-in operators, you can quickly and easily build queries and sophisticated real-time applications. Amazon Kinesis Data Analytics sets up the resources to run your applications and scales automatically to handle any volume of incoming data.
No Servers to Manage
You do not need to setup and manage a complex infrastructure for high availability and stateful processing. Amazon Kinesis Data Analytics is serverless and takes care of everything required to continuously run your application. This includes automatically provisioning the infrastructure to continuously process streaming data.
Automatic Elasticity with Pay-as-you-go Pricing
Amazon Kinesis Data Analytics elastically scales applications to keep up with any volume of data in the incoming data stream. You only pay for the resources used to run your streaming applications. You need not worry about provisioning infrastructure or paying for idle capacity.
Sub-Second Processing Latency
Amazon Kinesis Data Analytics delivers sub-second processing latencies so you can generate real-time alerts, dashboards, and actionable insights.
For SQL Users
Support for Standard SQL
Amazon Kinesis Data Analytics supports standard ANSI SQL, so all you need is familiarity with SQL.
Integrated Input and Output
Amazon Kinesis Data Analytics integrates with Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose so that you can readily ingest streaming data. Just point Amazon Kinesis Data Analytics at the input stream and it will automatically read the data, parse it, and make it available for processing. You can emit processed results to other AWS services including Amazon S3, Amazon Redshift, and Amazon Elasticsearch Service through Amazon Kinesis Data Firehose. You can also send output data to Amazon Kinesis Data Streams to build advanced stream processing pipelines.
Interactive SQL Editor
You get an interactive editor to build SQL queries using streaming data operations like sliding time-window averages. You can also view streaming results and errors using live data to debug or further refine your script interactively.
Easy-to-use Schema Editor
Amazon Kinesis Data Analytics provides an easy-to-use schema editor to discover and edit the structure of the input data. The wizard automatically recognizes standard data formats such as JSON and CSV. It infers the structure of the input data to create a baseline schema, which you can further refine using the schema editor.
Pre-built Stream Processing Templates
The interactive SQL editor comes bundled with a collection of stream processing templates that provide baseline SQL code for the most common types of operations such as aggregation, per-event transformation, and filtering. You simply select the template appropriate for your analytics task and then edit the provided code using the SQL editor to customize it for your specific use case.
Advanced Stream Processing Functions
Amazon Kinesis Data Analytics offers functions optimized for stream processing so that you can easily perform advanced analytics such as anomaly detection and top-K analysis on your streaming data.
For Java Users
Amazon Kinesis Data Analytics includes open source libraries based on Apache Flink. You can run them anywhere and there is no vendor lock-in. The libraries include Apache Flink, AWS SDK for Java, and AWS service integrations. Apache Flink is an open source framework and engine for building highly available and accurate streaming applications. The AWS SDK for Java helps take the complexity out of coding by providing Java APIs for many AWS services and includes the AWS Java library, code samples, and documentation.
Pre-built operators enable you to build a Java streaming application in hours instead of months. The Amazon Kinesis Data Analytics Java libraries are extensible and include more than 25 pre-built stream processing operators from Apache Flink like transform, partition, aggregate, join, and window to reduce your coding time and effort.
Integration with AWS Services
You can setup and integrate a data source or destination with minimal code. You can use the Amazon Kinesis Data Analytics Java libraries to integrate with Amazon S3, Amazon Managed Streaming for Apache Kafka (Amazon MSK), Amazon Elasticsearch Service, Amazon DynamoDB, Amazon Kinesis Data Streams, Amazon Kinesis Data Firehose, and Amazon CloudWatch.
Advanced Integration Capabilities
In addition to the AWS integrations, the Java libraries include more than ten connectors from Apache Flink and the ability to build custom integrations. With a couple more lines of code, you can modify how each integration behaves with advanced functionality. Also, you can build custom integrations using a set of Apache Flink primitives that enable you to read and write from files, directories, sockets, or other sources that you can access over the Internet.
Durable Application Backups
You can create and delete durable application backups through a simple API call. You can immediately restore your applications from the latest backup after a disruption, or you can restore your application to an earlier version.
Exactly Once Processing
Java applications in Amazon Kinesis Data Analytics enable you to build applications whose processed records affect the results exactly once, referred to as exactly once processing. This means that even in the case of an application disruption, like internal service maintenance or user initiated application update, the service will ensure that all data is processed and there is no duplicate data.
The service stores previous and in-progress computations, or state, in running application storage. This enables you to compare real-time and past results over any time period and provides fast recovery during application disruptions. State is always encrypted and incrementally saved in running application storage.