AWS IoT Core adds the ability to deliver data to Apache Kafka clusters

Posted on: Dec 15, 2020

AWS IoT Core now supports a new IoT rule action to deliver messages from your devices directly to your Amazon Managed Streaming for Apache Kafka (“Amazon MSK”) or self-managed Apache Kafka clusters for data analysis and visualization, without writing a single line of code.  

Because most Apache Kafka clusters run inside an Amazon Virtual Private Cloud (VPC), Apache Kafka action requires VPC delivery, and is used in conjunction with VPC action to deliver data to your VPC. Unlike other rule actions, VPC action does not require any configuration and is automatically enabled when you specify a VPC destination for your Apache Kafka action.

With Apache Kafka action and VPC action, you can create a rule to send electricity readings from electricity meter sensors of millions of households to an Apache Kafka cluster running on Amazon MSK or EC2 in your VPC for energy consumption analysis and reporting. 

Apache Kafka action integrates with AWS Secrets Manager, and supports Apache Kafka authentication methods such as username/password authentication, certificate-based authentication, and Kerberos authentication.  

Apache Kafka action is available in all AWS regions where AWS IoT Core is available. With this addition, AWS IoT Core now supports 21 action types. For more information about this feature, refer to the Apache Kafka action documentation.  

AWS IoT Core is a managed cloud service that lets connected devices easily and securely interact with cloud applications and other devices. AWS IoT Core can support billions of devices and trillions of messages, and can process and route those messages to AWS endpoints and to other devices reliably and securely. With AWS IoT Core, your applications can keep track of and communicate with all your devices, all the time, even when they aren’t connected. AWS IoT Core also makes it easy to use AWS and Amazon services like AWS Lambda, Amazon Kinesis, Amazon S3, Amazon Timestream, and Alexa Voice Service to build IoT applications that gather, process, analyze and act on data generated by connected devices, without having to manage any infrastructure.