The best way to get started with Amazon Kinesis Data Analytics is to get hands-on experience by building a sample application. Simply go to the Amazon Kinesis Data Analytics console and create a new Amazon Kinesis Data Analytics application. Depending on whether you choose a Java- or SQL-based application, use the following steps:
It's easy to get started with Java
To get started, create a Kinesis Data Analytics application that continuously reads and processes streaming data. Download the open source Java libraries using your favorite IDE, and then write your Java code and test it with live streaming data. You can configure destinations where you want Kinesis Data Analytics to send the results.
You can get instructions on how to download the libraries and create your first application in the Amazon Kinesis Data Analytics for Java Developer Guide.
Step 1: Download the open source libraries into your favorite IDE
You can start by downloading the open source libraries that include the AWS SDK, Apache Flink, and connectors for AWS services.
Step 2: Build a sample application in Java
You write your Java code using data streams and stream operators. Application data streams are the data structure you perform processing against using your Java code. Data continuously flows from the sources into application data streams. One or more stream operators are used to define your processing on the application data streams.
Step 3: Upload your code to Kinesis Data Analytics
Once built, upload your code to Amazon Kinesis Data Analytics and the service takes care of everything required to run your real-time applications continuously including scaling automatically to match the volume and throughput of your incoming data.
The Getting Started with Amazon Kinesis Data Analytics for Java Applications section of the Java Developer Guide provides a simple walkthough of building your first application.
Apache Flink provides several Java-based stream processing examples on Apache Flink GitHub repository.
The 15-minute training video explains how you use Java applications in Amazon Kinesis Data Analytics to get more timely insights from your data.
It's easy to get started with SQL
To get started, create a new Amazon Kinesis Data Analytics application. Select the demo stream we provide as input, pick a template, and edit the SQL query. You can then view the results right there in the console or load the output into Amazon Elasticsearch Service and visualize using Kibana. Within a few minutes, you will be able to deploy a complete streaming data application.
First, go to the Amazon Kinesis Data Analytics console and select a Kinesis data stream or Kinesis Data Firehose delivery stream as input. Amazon Kinesis Data Analytics ingests the data, automatically recognizes standard data formats, and suggests a schema. You can refine this schema, or if your input data is unstructured, you can define a new schema using our intuitive schema editor.
Next, write your SQL queries to process the streaming data using the Amazon Kinesis Data Analytics SQL editor and built-in templates, and test it with live streaming data.
Lastly, point to the destinations where you want the results loaded. Amazon Kinesis Data Analytics integrates out-of-box with Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose so it’s easy to send processed results to Amazon S3, Amazon Redshift, Amazon Elasticsearch Service, or your own custom destination.
Getting started examples
These resources provide example streaming data applications and step-by-step instructions so you can try them out and gain hands-on experience.
This SQL Developer Guide gives you an overview of the Amazon Kinesis Data Analytics architecture, creating applications, and confuguring inputs and outputs.
In the Getting Started guide, we step you through setting up an AWS Account, the Command Line Interface (AWS CLI), and creating your starter Amazon Kinesis Data Analytics application.
It's easy to get started with Kinesis Data Analytics. The how-to videos make it even easier by providing technical deep dives into common use cases and stream processing workflows. They also provide in-depth overviews of key features so you can get your job done. Follow the links below to watch the recordings: