Developer guide: This guide provides a conceptual overview of Amazon Kinesis Data Streams and includes detailed instructions for using various features of the service.
API reference: This is a detailed reference guide that describes all the API operations of Amazon Kinesis Data Streams, along with sample requests, responses, etc.
Tools and libraries
Amazon Kinesis Agent
This is a pre-built Java application that offers an easy way to collect and send data to your Amazon Kinesis data stream. You can install the agent on Linux-based server environments such as web servers, log servers, and database servers. The agent monitors certain files and continuously sends data to your data stream. Get Amazon Kinesis Agent.
Amazon Kinesis Client Library
This is a pre-built library that helps you easily build Amazon Kinesis Applications for reading and processing data from an Amazon Kinesis stream. This library handles complex issues such as adapting to changes in stream volume, load-balancing streaming data, coordinating distributed services, and processing data with fault-tolerance, enabling you to focus on business logic while building applications. Get Kinesis Client Library for Java | Python | Ruby | Node.js | .NET
AWS Streaming Data Solution for Amazon Kinesis
Use AWS Streaming Data Solution for Amazon Kinesis to help you solve for real-time streaming use cases like capturing high volume application logs, analyzing clickstream data, continuously delivering to a data lake, and more.
This solution shortens your development time by days by removing or reducing the need for you to: model and provision resources using AWS CloudFormation; set up Amazon CloudWatch alarms, dashboards, and logging; and manually implement streaming data best practices in AWS. This solution is data and logic agnostic, enabling you to start with boilerplate code and quickly start customizing. After deployment, you can use the solution’s monitoring capabilities to easily transition to production.
Amazon Kinesis Producer Library
This is an easy to use and highly configurable library that helps you put data into an Amazon Kinesis stream. This library presents a simple, asynchronous, and reliable interface that enables you to quickly achieve high producer throughput with minimal client resources. Get Kinesis Producer Library.
Amazon Kinesis Connector Library
This is a pre-built library that helps you easily integrate Amazon Kinesis Data Streams with other AWS services and third-party tools. Amazon Kinesis Client Library (KCL) is required for using this library. The current version of this library provides connectors to Amazon DynamoDB, Amazon Redshift, Amazon S3, and Elasticsearch. The library also includes sample connectors of each type, plus Apache Ant build files for running the samples. Get Kinesis Connector Library.
Amazon Kinesis Storm Spout
This is a pre-built library that helps you easily integrate Amazon Kinesis Data Streams with Apache Storm. The current version of this library fetches data from Amazon Kinesis stream and emits it as tuples. You will add the spout to your Storm topology to leverage Amazon Kinesis Data Streams as a reliable, scalable, stream capture, storage, and replay service. Get Kinesis Storm Spout.
Related AWS tools and libraries
Amazon EMR Connector for Kinesis
Enables Amazon Elastic MapReduce (EMR) to directly read and query data from Amazon Kinesis data streams. Learn more | Frequently Asked Questions for EMR Connector to Kinesis
AWS Mobile SDK Integration
Log4J Appender for Amazon Kinesis Data Streams
An implementation of the Apache Log4J Appender Interface that pushes Log4J output directly to Amazon Kinesis data streams without requiring any custom code. Learn more
Community contributed tools and libraries
Fluentd plugin for Amazon Kinesis Data Streams
A plugin that allows Fluentd to add data to Amazon Kinesis data streams. Learn more
Amazon Kinesis Source and Sink for Apache Flume
Amazon Kinesis Source and Sink for Apache Flume service. Learn more
Spark Streaming and Amazon Kinesis Data Streams Integration
An Amazon Kinesis Data Streams receiver that creates an input DStream using the Amazon Kinesis Client Library. Learn more
Amazon Kinesis Client Library for Go Developers
Enables developers to write record processors in Go language using Amazon Kinesis Client Library multi-language daemon. Learn more
Amazon Kinesis Data Streams Aggregators
Enables the automatic creation and visualization of aggregated time series data from Amazon Kinesis data streams. Learn more
Amazon Kinesis Data Streams Scaling Utility
Enables scaling the number of shards of an Amazon Kinesis data stream. Developers can scale up or down by specifying a certain number of shards or a percentage of the total number of shards. Learn more
Amazon Kinesis Data Streams Module for Vert.x
Enables receiving events published by other Vert.x verticles and sending those events to Amazon Kinesis Data Streams. Learn more