Ingesting industrial media to Amazon Kinesis Video Streams using AWS IoT Greengrass V2 components

Introduction

Organizations install hundreds of Internet Protocol (IP) cameras to increase security by surveilling indoor and outdoor spaces. This is a common need for manufacturing plant floors across industries such as automotive, commercial, Oil & Gas, public safety, and agri-tech. Companies connect cameras to the cloud to create a centralized view of their siloed data and also to add digital twin capabilities to their facilities. In this post, we will discuss how to use AWS IoT Greengrass V2 components to package and deploy an Amazon Kinesis Video Streams stream uploader that can do live-streaming, on-demand video upload, and local caching, thus facilitating use cases that require ingestion of live audio and video streams from pre-installed IP cameras.

Kinesis Video Streams is a fully managed AWS service that you can use to stream live video from devices to AWS, build applications for real-time video processing, perform batch video analytics, and more. You can use the Kinesis Video Streams service to ingest video and non-video data from many different sources like smartphones, security cameras, web cameras, drones, thermal imagery, and audio. Recently, we released an AWS IoT Greengrass component for Kinesis Video Streams that lets you stream media from your existing devices. The edge connector for the Kinesis Video Streams component (aws.iot.EdgeConnectorForKVS) reads video feeds from local IP cameras using Real Time Streaming Protocol (RTSP) and publishes the streams to the Kinesis Video Streams endpoint.

Solution overview

A company that develops smart building solutions is interested in building an application that ingests hundreds of video streams from the building entry, access control areas, and security gates. The company might be looking to incorporate video feeds in to a digital twin application using AWS IoT TwinMaker, along with using the AWS IoT TwinMaker application plugin for Grafana dashboard to request uploading of videos and checking historical video timelines.

It can be challenging to refactor the existing cameras to stream to an endpoint. Instead, you can deploy an AWS IoT Greengrass edge gateway and an AWS IoT Greengrass edge connector for Kinesis Video Streams component to ingest data from these cameras. The component connects to IP cameras within the same network and streams the video feed to Kinesis Video Streams. On the consumption side, you can use an application to read from the Kinesis Video Streams endpoint to act as a client. This component supports features like edge video caching, scheduled video recording, scheduled video uploading, live streaming, and historical video uploading to Kinesis Video Streams. The edge connector component provides a fully working AWS IoT Greengrass V2 component for video ingestion that can be customized based on needs.

The number of cameras that can be connected to this component per AWS IoT Greengrass hub is dependent on the compute power of the underlying hardware, network bandwidth, and AWS IoT SiteWise child assets used to store the configuration of the connector (currently this limit is 2000, refer to the documentation for details on child asset quotas). This architecture assumes that there is a stable network connection between the AWS IoT Greengrass device and AWS, with sufficient bandwidth for streaming the media.

Figure: Ingesting video feed from IP cameras using AWS IoT Greengrass v2 components

1. Deployment of AWS IoT Greengrass Core at the edge device. This device will be responsible for running the edge connector and interfacing with the cameras. The Greengrass Core software can be deployed on a Linux device, such as a Raspberry Pi or a Windows device. This device will eventually run the edge connector  Kinesis Video Streams component. Refer documentation for more details on how to setup AWS IoT Greengrass.
2. Install GStreamer version 1.18.4 or later on the edge device.
3. Once the edge device is setup, use the AWS IoT Greengrass service to deploy the edge connector for Kinesis Video Streams component. Edit the configuration page with the details specific to your deployment. Refer documentation to learn more about how to deploy components to AWS IoT Greengrass.
4. Once the edge connector for Kinesis Video Streams component is deployed, the configuration for the component is stored in AWS IoT SiteWise and AWS Secrets Manager. AWS IoT SiteWise stores two types of assets: theEdgeConnectorForKVSHub asset contains the asset name that identifies the unique hub where the connector is running and the EdgeConnectorForKVSCamera contains the properties specific to the cameras, like cron expression to start streaming and recording. For details, refer the GitHub page about the configuration parameters needed for this service .
5. The edge connector for Kinesis Video Streams ingests data from the camera feeds. There is an option here to add local storage as well as stream them to the Kinesis Video Streams endpoint.
6. On the client side, you can build your own custom applications to consume data from the Kinesis Video Streams endpoint. As an example, you can trigger live streaming video when you detect a motion.

For detailed steps on implementing this architecture and above steps, refer to github documentation.

Detailed edge architecture

Figure: Architecture for ingesting IP video cameras feed in Amazon Kinesis Video Stream

The edge architecture has three modules: controller, video recorder, and video uploader (see the previous diagram). While the default setting of the Kinesis Video Streams connector component is to stream the video, it has an optional functionality of recording video on the file system for local storage. The controller acts as a broker between the recorder and the uploader. It also facilitates communication between the two. The controller first initiates a pair of piped input and output stream objects. The video recorder retrieves stream data from the camera and puts the data into the piped output stream. Finally, the video uploader takes stream data from the piped input stream, then uploads the data to Kinesis Video Streams.

Scaling the solution

Next, we’ll look at sizing and limits to see how the solution scales. In the architecture, the edge connector for Kinesis Video Streams component and Greengrass Core do not have any scaling limitations. As the solution uses AWS IoT SiteWise to manage the RTSP camera configuration, the only hard limit is from the AWS IoT SiteWise child assets quota, which is fewer than 2000 child assets per parent asset. The number of cameras that can be supported by the edge device/Hub only depends on its hardware configurations. If there is enough network bandwidth and hardware capacity, the AWS IoT Greengrass device can support more cameras. From our internal testing, we tested 10+ cameras connected to the same edge device to ingest the feeds without any issues. Refer to the documentation Kinesis Video Streams API limits and quotas.

Below are some sample edge device configurations and the number of video streams they can support for optimal performance:

1. A small instance (like Raspberry Pi 4 Model B) with 2GB RAM and 16GB SSD can support up to 2 1080p HD RTSP cameras uploading to the cloud at the same time with a network speed of 100 MBPS.
2. A medium instance (like NVIDIA Jetson Nano Developer Kit) with 4GB RAM and 16GB SSD can support up to 4 1080p HD RTSP cameras uploading to the cloud at the same time over a network speed of 100 MBPS.
3. A large instance (like Intel NUC) with 25GB RAM and 1T SSD can support up to 24 1080p HD RTSP cameras uploading to the cloud at the same time with a network speed of 600 MBPS.

This solution is primarily memory dependent, hence compute resources like CPU and GPU type and capacity are less relevant.

Clean up

If you used the GitHub link to implement this architecture, make sure to use the below steps to clean-up the resources to avoid incurring cost.

1. Uninstall Greengrass core software from the edge device
2. Delete Kinesis Video Stream:
   1. Open the Kinesis Video Streams console
   2. Choose Video streams in the left-hand menu and select the video stream
   3. Choose Delete video stream in the upper right corner of the screen
   4. A confirmation screen will appear. Enter Delete in the field and select Delete.

Conclusion

In this blog post, we have given an overview of the AWS IoT Greengrass component for Kinesis Video Streams, use cases where it can be deployed, and the GitHub link for steps to deploy this connector. You can use this feature just like any other AWS-provided Greengrass component and enable Kinesis Video Streams for your existing devices. To learn more about this solution in detail including steps to deploy, please refer this git hub repository.

Authors