Documentation

Developer guide: This guide provides a conceptual overview of Amazon Kinesis Data Streams and includes detailed instructions for using various features of the service.

API reference: This is a detailed reference guide that describes all the API operations of Amazon Kinesis Data Streams, along with sample requests, responses, etc.

Videos

AWS re:Invent 2020: How Disney+ uses fast data ubiquity to improve the customer experience (30:34)
High Performance Data Streaming with Amazon Kinesis: Best Practices and Common Pitfalls (38:05)
AWS re:Invent 2020: Top 5 best practices for data streaming with Amazon Kinesis (27:47)
Streaming Data Pipelines for Real-Time Analytics – Are You Ready? (40:04)

Blog posts

No blog posts have been found at this time. Please see the AWS Blog for other resources.

Tools and libraries

Amazon Kinesis Agent

This is a pre-built Java application that offers an easy way to collect and send data to your Amazon Kinesis data stream. You can install the agent on Linux-based server environments such as web servers, log servers, and database servers. The agent monitors certain files and continuously sends data to your data stream. Get Amazon Kinesis Agent.

Amazon Kinesis Client Library

This is a pre-built library that helps you easily build Amazon Kinesis Applications for reading and processing data from an Amazon Kinesis stream. This library handles complex issues such as adapting to changes in stream volume, load-balancing streaming data, coordinating distributed services, and processing data with fault-tolerance, enabling you to focus on business logic while building applications. Get Kinesis Client Library for Java | Python | Ruby | Node.js | .NET

AWS Streaming Data Solution for Amazon Kinesis

Use AWS Streaming Data Solution for Amazon Kinesis to help you solve for real-time streaming use cases like capturing high volume application logs, analyzing clickstream data, continuously delivering to a data lake, and more.

This solution shortens your development time by days by removing or reducing the need for you to: model and provision resources using AWS CloudFormation; set up Amazon CloudWatch alarms, dashboards, and logging; and manually implement streaming data best practices in AWS. This solution is data and logic agnostic, enabling you to start with boilerplate code and quickly start customizing. After deployment, you can use the solution’s monitoring capabilities to easily transition to production.

Amazon Kinesis Producer Library

This is an easy to use and highly configurable library that helps you put data into an Amazon Kinesis stream. This library presents a simple, asynchronous, and reliable interface that enables you to quickly achieve high producer throughput with minimal client resources. Get Kinesis Producer Library.

Amazon Kinesis Connector Library

This is a pre-built library that helps you easily integrate Amazon Kinesis Data Streams with other AWS services and third-party tools. Amazon Kinesis Client Library (KCL) is required for using this library. The current version of this library provides connectors to Amazon DynamoDB, Amazon Redshift, Amazon S3, and Elasticsearch. The library also includes sample connectors of each type, plus Apache Ant build files for running the samples. Get Kinesis Connector Library.

Amazon Kinesis Storm Spout

This is a pre-built library that helps you easily integrate Amazon Kinesis Data Streams with Apache Storm. The current version of this library fetches data from Amazon Kinesis stream and emits it as tuples. You will add the spout to your Storm topology to leverage Amazon Kinesis Data Streams as a reliable, scalable, stream capture, storage, and replay service. Get Kinesis Storm Spout.

Learn how to build a real-time sliding-window dashboard with Amazon Kinesis Data Streams and Apache Storm.

Amazon EMR Connector for Kinesis

Enables Amazon Elastic MapReduce (EMR) to directly read and query data from Amazon Kinesis data streams. Learn more | Frequently Asked Questions for EMR Connector to Kinesis

AWS Mobile SDK Integration

Amazon Kinesis Recorder enables you to reliably record data to an Amazon Kinesis data stream from your mobile application. Learn more for iOS | Learn more for Android

Log4J Appender for Amazon Kinesis Data Streams

An implementation of the Apache Log4J Appender Interface that pushes Log4J output directly to Amazon Kinesis data streams without requiring any custom code. Learn more

Community contributed tools and libraries

Fluentd plugin for Amazon Kinesis Data Streams

A plugin that allows Fluentd to add data to Amazon Kinesis data streams. Learn more

Amazon Kinesis Source and Sink for Apache Flume

Amazon Kinesis Source and Sink for Apache Flume service. Learn more

Spark Streaming and Amazon Kinesis Data Streams Integration

An Amazon Kinesis Data Streams receiver that creates an input DStream using the Amazon Kinesis Client Library. Learn more

Amazon Kinesis Client Library for Go Developers

Enables developers to write record processors in Go language using Amazon Kinesis Client Library multi-language daemon. Learn more

Amazon Kinesis Data Streams Aggregators

Enables the automatic creation and visualization of aggregated time series data from Amazon Kinesis data streams. Learn more

Amazon Kinesis Data Streams Scaling Utility

Enables scaling the number of shards of an Amazon Kinesis data stream. Developers can scale up or down by specifying a certain number of shards or a percentage of the total number of shards. Learn more

Amazon Kinesis Data Streams Module for Vert.x

Enables receiving events published by other Vert.x verticles and sending those events to Amazon Kinesis Data Streams. Learn more

Get started with Amazon Kinesis Data Streams

See the developer guide
Ready to get started?
Sign up
Have more questions?
Contact us