AWS Greengrass Machine Learning Inference

AWS IoT Greengrass makes it easy to perform machine learning inference locally on devices, using models that are created, trained, and optimized in the cloud. AWS IoT Greengrass gives you the flexibility to use machine learning models trained in Amazon SageMaker or to bring your own pretrained model stored in Amazon S3.

Machine learning uses statistical algorithms that learn from existing data, a process called training, in order to make decisions about new data, a process called inference. During training, patterns and relationships in the data are identified to build a model. The model allows a system to make intelligent decisions about data it hasn’t encountered before. Optimizing models compresses the model size so it runs quickly. Training and optimizing machine learning models require massive computing resources, so it is a natural fit for the cloud. But, inference takes a lot less computing power and is often done in real-time when new data is available. Getting inference results with very low latency is important to ensure your IoT applications can respond quickly to local events.

AWS IoT Greengrass gives you the best of both worlds. You use machine learning models that are built, trained, and optimized in the cloud and run inference locally on devices. For example, you can build a predictive model in SageMaker for scene detection analysis, optimize it to run on any camera, and then deploy it to predict suspicious activity and send an alert. Data gathered from the inference running on AWS IoT Greengrass can be sent back to SageMaker where it can be tagged and used to continuously improve the quality of machine learning models.

Benefits

Flexible

AWS IoT Greengrass includes prebuilt Amazon SageMaker Neo Deep Learning Runtime (DLR), Apache MXNet, TensorFlow, and Chainer packages for devices powered by Intel Atom, NVIDIA Jetson TX2, and Raspberry Pi so you don’t have to build and configure the machine learning framework for your devices from scratch. In addition, it works with other popular frameworks including PyTorch and Caffe2. If you use Amazon SageMaker Neo with AWS IoT Greengrass, models written in these framework are converted into portable code that will run on any AWS IoT Greengrass device that includes the Neo runtime, so you don’t have to do additional tuning at the edge.

Deploy models to your connected devices with a few clicks

AWS IoT Greengrass makes it easy to deploy your machine learning model from the cloud to your devices. With just a few clicks in the AWS IoT Greengrass console, you can locate trained models in Amazon SageMaker or Amazon S3, select the desired model, and deploy it to the target devices. Your models will be deployed on the connected device of your choice.

Accelerate inference performance

Through the integration with Amazon SageMaker and the Neo deep learning compiler, you can deploy machine learning models with an optimized runtime that runs up to twice as fast compared to hand tuning or using machine learning frameworks. AWS IoT Greengrass also gives you access to hardware accelerators, such as GPUs on your devices, by supplying prebuilt runtimes for the common machine learning frameworks and target devices, such as the Nvidia Jetson TX2 board.

Run inference on more devices

Using integration with Amazon SageMaker and the Neo compiler, models are optimized with less than a tenth of the memory footprint so they can run on resource constrained devices such as home security cameras and actuators.

Easily run inference on connected devices

Performing inference locally on devices running AWS IoT Greengrass, reduces the latency and cost of sending device data to the cloud to make a prediction. Rather than sending all data to the cloud for performing machine learning inference, you run inference directly on the device.

Build more accurate models

Using AWS IoT Greengrass, you can perform inference and capture the results, detect outliers, and send data back to the cloud and Amazon SageMaker where it can be reclassified, tagged, and used for model retraining to improve the accuracy of your machine learning model.

How it works

Use cases

Predictive industrial maintenance

As pricing pressure increases on manufacturers, they are looking for newer ways to help increase operational efficiency on factory floors. Delays in detecting issues on the manufacturing assembly line can lead to a waste of time and resources. AWS IoT Greengrass can help you in early detection of faulty equipment and issues on the factory floor. IoT Greengrass-powered industrial gateways can continuously monitor sensor data (e.g., vibrations, noise-level), predict anomalies, and take relevant actions such as send alerts or stop equipment to minimize losses.

Precision agriculture

The agriculture industry is going through two major disruptions. First, the world’s population continues to grow, causing the demand for food to outweigh the output. Second, climate change is resulting in unpredictable weather conditions, affecting crop yields. AWS IoT Greengrass can help transform agriculture practices and deliver new value to customers. AWS IoT Greengrass-powered cameras installed in greenhouses and farms can process images of plants, crops, and data from sensors in the soil to not only detect environmental anomalies such as change in temperature, moisture, and nutrition level, but also trigger alerts.

Security

Security camera manufacturers are looking for new ways to make devices more intelligent and automate their threat detection capabilities. AWS IoT Greengrass can help improve the capabilities of security cameras. AWS IoT Greengrass-enabled cameras can continuously scan premises to look for a change in the scene, such as an incoming visitor, and send an alert. The cameras are able to quickly perform scene detection analyses locally and send data to the cloud only when required.

Retail and hospitality

Retailers, cruise lines, and amusement parks are investing in IoT applications to provide better customer service. For example, you can run object detection models at amusement parks to keep track of visitor count. Cameras locate the visitors and maintain a running headcount locally without having to send massive amounts of video feed to the cloud, which is often a challenge due to limited internet bandwidth. This solution can predict wait times at popular theme park rides and help improve the customer experience.

Video processing

AWS IoT Greengrass can be deployed on connected devices like security cameras, traffic cameras, body cameras, and medical imaging equipment to help them make predictions locally. With AWS IoT Greengrass, you can deploy and run machine learning models like facial recognition, object detection, and image density directly on the device. For example, a traffic camera could count bicycles, vehicles, and pedestrians passing through an intersection and detect when traffic signals need to be adjusted in order to optimize traffic flows and keep people safe.

Featured Customers

AWS IoT Greengrass helps Yanmar increase the intelligence of greenhouse operations by automatically detecting and recognizing the main growth stages of vegetables to grow more crops.

The Electronic Caregiver ensures high-quality caregiving with AWS IoT Greengrass ML Inference and can push machine learning models directly to edge devices and keep patients safer.

With AWS IoT Greengrass, Vantage Power pushes machine learning models to individual vehicles and detects battery faults 1 month earlier.

AWS IoT Greengrass ML Inference