Amazon Kinesis Data Analytics
Amazon Kinesis Data Analytics is the easiest way to transform and analyze streaming data in real time with Apache Flink. Apache Flink is an open source framework and engine for processing data streams. Amazon Kinesis Data Analytics reduces the complexity of building, managing, and integrating Apache Flink applications with other AWS services.
Amazon Kinesis Data Analytics takes care of everything required to run streaming applications continuously, and scales automatically to match the volume and throughput of your incoming data. With Amazon Kinesis Data Analytics, there are no servers to manage, no minimum fee or setup cost, and you only pay for the resources your streaming applications consume.
Powerful real-time processing
Amazon Kinesis Data Analytics provides built-in functions to filter, aggregate, and transform streaming data for advanced analytics. It processes streaming data with sub-second latencies, enabling you to analyze and respond to incoming data and events in real time.
No servers to manage
Amazon Kinesis Data Analytics is serverless; there are no servers to manage. It runs your streaming applications without requiring you to provision or manage any infrastructure. Amazon Kinesis Data Analytics automatically scales the infrastructure up and down as required to process incoming data.
Pay only for what you use
With Amazon Kinesis Data Analytics, you only pay for the processing resources that your streaming applications use. There are no minimum fees or upfront commitments.
Easy to use
Amazon Kinesis Data Analytics enables you to easily and quickly build queries and sophisticated streaming applications in three simple steps: setup your streaming data sources, write your queries or streaming applications, and setup your destination for processed data.
Build sophisticated streaming applications with Apache Flink
Amazon Kinesis Data Analytics includes open source libraries and runtimes based on Apache Flink that enable you to build an application in hours instead of months using your favorite IDE. The extensible libraries include specialized APIs for different use cases, including stateful stream processing, streaming ETL, and real-time analytics. You can use the libraries to integrate with AWS services like Amazon Managed Streaming for Apache Kafka (Amazon MSK), Amazon Kinesis Data Streams, Amazon Kinesis Data Firehose, Amazon Elasticsearch Service, Amazon S3, Amazon DynamoDB, and more.
Use your favorite language
Amazon Kinesis Data Analytics supports building applications in SQL, Java, Scala, and Python. You can use these languages to easily build applications that perform joins, aggregations over time windows, filters, and more. You can extend the open source libraries and include custom libraries from the language of your choice. Using Amazon Kinesis Data Analytics Studio, you can build applications interactively in SQL, Scala, and Python using Apache Zeppelin notebooks.
Quickly get started using Amazon Kinesis Data Analytics Studio
With Amazon Kinesis Data Analytics Studio, you can interactively query data streams and rapidly develop stream processing applications using an interactive development environment powered by Apache Zeppelin notebooks. With Amazon Kinesis Data Analytics Studio, streaming data processing is powered by Apache Flink.
Build streaming applications with Apache Beam
Amazon Kinesis Data Analytics supports running streaming applications built through Apache Beam’s Java SDK in a serverless Apache Flink environment. Apache Beam is an open-source, unified model for defining streaming and batch data processing applications that can be executed across multiple execution engines. You can easily build Apache Beam streaming applications in Java and run them on Amazon Kinesis Data Analytics and other execution engines.
How it works
You can develop streaming extract-transform-load (ETL) applications with Amazon Kinesis Data Analytics built-in operators to transform, aggregate, and filter streaming data. You can easily deliver your data in seconds to Amazon Kinesis Data Streams, Amazon Managed Streaming for Apache Kafka (Amazon MSK), Amazon Elasticsearch Service, Amazon S3, custom integrations, and more using built-in connectors.
Try out Streaming ETL with Apache Flink and Amazon Kinesis Data Analytics using sample code on GitHub.
You can interactively query and analyze data streams in real time and continuously produce insights and results for time sensitive use cases such as click stream analytics.
Stateful event processing
You can develop applications that process events from one or more data streams and trigger conditional processing and external actions. You can identify patterns like anomaly detection in your data streams using standard SQL and Apache Flink libraries for complex event processing.
Check out how to enrich your data stream using Amazon Kinesis Data Analytics.
Get started with Amazon Kinesis Data Analytics
Instantly get access to the AWS Free Tier.
Build your streaming application from the Amazon Kinesis Data Analytics console.