Q. What is AWS IoT Analytics?
AWS IoT Analytics is a fully-managed IoT analytics service that collects, pre-processes, enriches, stores, and analyzes IoT device data at scale. IoT Analytics can perform simple ad hoc queries as well as complex analysis, and is a simpler way to run IoT analytics for use cases such as understanding the performance of devices, predicting device failures, and machine learning. It is designed specifically for IoT and automatically captures and stores the message timestamp so it is easy to perform time-series analysis. IoT Analytics can also enrich the data with device-specific metadata such as device type and location using the AWS IoT registry. IoT Analytics stores data in an IoT-optimized data store so you can run queries on large datasets.
Q. How does AWS IoT Analytics work?
AWS IoT Analytics is fully integrated with AWS IoT Core so it is easy to get started. First, you define a channel and select the data you want to collect so you only store and analyze the data of interest, such as sensor temperature. Once the channel is set up, you configure pipelines to process your data. Pipelines support transformations like Celsius to Fahrenheit conversion, conditional statements, message filtering, and message enrichment using external data sources and AWS Lambda functions.
After processing the data in the pipeline, IoT Analytics stores it in an IoT-optimized data store for analysis. You can query the data using the built-in SQL query engine to answer specific business questions. For example, you may want to know how many monthly active users there are for each device in your fleet. Through integration with Amazon SageMaker, IoT Analytics supports more sophisticated analytics, like Bayesian inference and machine learning. Also, it is easy to build visualizations and dashboards and get business insights quickly from your IoT Analytics data since it is integrated with Amazon QuickSight.
Q. What retention policies do I’ve on my Data Stores and Channels?
You can choose to set your own data retention policies on the Data stores and Channels. You can select the retention period for any number of days. If you do not select any retention period, the service will retain your data for unlimited period.
Q. What type of message formats are supported with AWS IoT Analytics?
AWS IoT Analytics service supports JSON and Binary formats (Binary JSON, protocol buffers, etc.). In case the messages are binary, the first activity of your Pipeline needs to be the Lambda activity with an output of a valid JSON.
Q. Can I re-process my data from Channel to a Pipeline?
Yes, you can re-process your data from the Channel connected to the Pipeline using StartPipelineReprocessing. If you want to re-process your Channel data using another Pipeline, you will need to connect the Pipeline to your appropriate Channel.
Q. How do I get the data into AWS IoT Analytics using the Ingestion API?
You can use the BatchPutMessage API to send your data to AWS IoT Analytics from sources like Amazon S3, Amazon Kinesis or any other data source. You can use this APIs within your Lambda function or any other script to send the data to AWS IoT Analytics. For more information, please refer to Send data from S3 to IoT Analytics and send data from Kinesis to IoT Analytics.
Q. Can I preview my messages in the Channel?
Yes, you can preview your messages using SampleChannelData API. This API samples up to ten messages from your Channel across all data.
Q. Can I simulate my pipeline activity?
Yes, you can use the RunPipelineActivity API or console to simulate your pipeline activity. This API takes the list of sample data (either from SampleChannelData or supplied manually) to simulate the activity and previews the results.
Q. What are the differences between AWS IoT Analytics and Amazon Kinesis Analytics?
AWS IoT Analytics is designed specifically for IoT and automatically captures and stores the message timestamp so it is easy to perform time-series analytics. IoT Analytics can also enrich the data with device-specific metadata such as device type and location using the AWS IoT registry and other public data sources. IoT Analytics stores the device data in IoT-optimized data store so you can run queries on large datasets.
Amazon Kinesis Analytics is a general-purpose tool designed to easily process streaming data from IoT devices as well as other data sources in real time.
Table 1: AWS IoT Analytics vs. Kinesis Analytics feature comparison
|Features||AWS IoT Analytics
||Amazon Kinesis Analytics|
|Storage of time-series data||X|
|Automatic data partitions by message timestamp and device ID||X|
|Device-specific data enrichment||X|
|Queries on large datasets||X|
|Real-time processing||Minutes or seconds latencies||Seconds or milliseconds latency|
|Parse unstructured data and automatically create schema||JSON and CSV||JSON and CSV|
Q. When do I use AWS IoT Analytics and when do I use Amazon Kinesis Analytics?
You can use AWS IoT Analytics for IoT analytics. Some use cases include understanding long-term device performance, business reporting and ad-hoc analysis, and predictive fleet maintenance. IoT Analytics is best suited for these use cases because it collects, prepares, and stores data from devices over long timeframes in an IoT-optimized data store. IoT Analytics also enriches the data with device-specific metadata such as device type and location using the AWS IoT registry and other public data sources.
However, if you need to analyze IoT data in real-time for use cases such as device monitoring, you can use Amazon Kinesis Analytics.
Table 2: AWS IoT Analytics vs. Kinesis Analytics use cases
||AWS IoT Analytics||Amazon Kinesis Analytics
|Understanding Long-Term Device Performance Characteristics||Yes. Enrich IoT data with IoT-specific metadata such as device type and location using AWS IoT registry and other public data sources. For example, vineyard operators need to enrich humidity sensor data with expected rainfall at the vineyards so they know when to water crops.||No. Best suited for real-time, streaming analytics.|
|Business Reporting and Ad-Hoc Analysis on IoT Data||Yes. Collect, process, and store IoT data and integrate with AWS QuickSight to build dashboards and reporting or use built-in SQL query engine for ad-hoc queries. For example, aggregate sensor failures across a fleet to report on fleet performance every week.||No. Best suited to perform streaming queries on IoT data, such as generating alerts when a sensor fails.|
|Predictive Fleet Maintenance||Yes. Collect, process, and store IoT data and use pre-built templates to build and deploy predictive models. For example, predicting when HVAC systems will fail on connected vehicles so the vehicle can be rerouted and docking expedited to prevent shipment damage.||No. Predictive maintenance requires a historical analysis on long-term data to build models.|
|Real-Time Device Monitoring||No.||Yes. Kinesis Analytics can aggregate data over time windows continuously, detect anomalies, and take actions such as sending alerts. For example, Kinesis Analytics can calculate rolling 10-second averages of valve temperatures every 5 minutes in industrial equipment, and detect when the temperature exceeds certain preset thresholds. It can then alert control systems to automatically shut off machinery, avoiding accidents.|
Q. When do I use AWS IoT Analytics and Amazon Kinesis together?
Use AWS IoT Analytics and Amazon Kinesis together when you need both historical and real-time analytics. For example, use Kinesis Analytics to calculate 10-second rolling averages of valve temperatures in industrial equipment to detect when the temperature exceeds certain thresholds. Kinesis Analytics can then alert control systems to automatically shut off machinery, avoiding accidents. Then, use Kinesis Streams to send data to IoT Analytics. You use IoT Analytics to understand trends and also predict when valves should be replaced or serviced.
Q. When working with IoT data, when should I use AWS IoT Analytics vs. Amazon Kinesis Streams, Amazon Kinesis Analytics and Amazon Kinesis Firehose?
Customers can use AWS Lambda to send data from Amazon Kinesis Streams to an AWS IoT data channel and then to AWS IoT Analytics.
Amazon Kinesis Analytics is designed for streaming analytics, while IoT Analytics is designed for analytics on data at rest. Customers who need both real-time and IoT analytics can use a combination of Kinesis Analytics and IoT Analytics.
Amazon Kinesis Firehose is the easiest way to load streaming data into AWS data stores Amazon S3, Amazon Redshift, and Amazon Elasticsearch Service, enabling near real-time analytics with existing business intelligence tools. IoT Analytics does not support Kinesis Firehose as a data source.
Q. When do I use AWS IoT Analytics and when do I use Amazon Kinesis Video Streams?
Amazon Kinesis Video Streams makes it easy to securely stream video, audio and other time-encoded data from sources like cameras, depth sensors, and RADARs to AWS for real-time and batch processing in machine learning, analytics, and other applications. Kinesis Video Streams is particularly designed for ingesting video data from devices, whereas AWS IoT Analytics is designed for analyzing non-video IoT data. At GA, there is no direct integration between Kinesis Video Streams and AWS IoT Core or IoT Analytics. However, customers can query Kinesis Video Streams and IoT Analytics from their applications using the APIs.