Why IoT Greengrass ML inference?
AWS IoT Greengrass makes it easy to perform machine learning (ML) inference locally on devices, using models that are created, trained, and optimized in the cloud. AWS IoT Greengrass gives you the flexibility to use machine learning models trained in Amazon SageMaker or to bring your own pretrained model stored in Amazon S3.
Machine learning uses statistical algorithms that learn from existing data, a process called training, in order to make decisions about new data, a process called inference. During training, patterns and relationships in the data are identified to build a model. The model allows a system to make intelligent decisions about data it hasn’t encountered before. Optimizing models compresses the model size so it runs quickly. Training and optimizing machine learning models require massive computing resources, so it is a natural fit for the cloud. But, inference takes a lot less computing power and is often done in real-time when new data is available. Getting inference results with very low latency is important to ensure your IoT applications can respond quickly to local events.
AWS IoT Greengrass gives you the best of both worlds. You use machine learning models that are built, trained, and optimized in the cloud and run inference locally on devices. For example, you can build a predictive model in SageMaker for scene detection analysis, optimize it to run on any camera, and then deploy it to predict suspicious activity and send an alert. Data gathered from the inference running on AWS IoT Greengrass can be sent back to SageMaker where it can be tagged and used to continuously improve the quality of machine learning models.
Benefits
Flexible
AWS IoT Greengrass includes prebuilt Amazon SageMaker Neo Deep Learning Runtime (DLR), Apache MXNet, TensorFlow, and Chainer packages for devices powered by Intel Atom, NVIDIA Jetson TX2, and Raspberry Pi so you don’t have to build and configure the machine learning framework for your devices from scratch. In addition, it works with other popular frameworks including PyTorch and Caffe2. If you use Amazon SageMaker Neo with AWS IoT Greengrass, models written in these framework are converted into portable code that will run on any AWS IoT Greengrass device that includes the Neo runtime, so you don’t have to do additional tuning at the edge.
Deploy models to your connected devices in a few quick steps
AWS IoT Greengrass makes it easy to deploy your machine learning model from the cloud to your devices. With just a few clicks in the AWS IoT Greengrass console, you can locate trained models in Amazon SageMaker or Amazon S3, select the desired model, and deploy it to the target devices. Your models will be deployed on the connected device of your choice.
Accelerate inference performance
Through the integration with Amazon SageMaker and the Neo deep learning compiler, you can deploy machine learning models with an optimized runtime that runs up to twice as fast compared to hand tuning or using machine learning frameworks. AWS IoT Greengrass also gives you access to hardware accelerators, such as GPUs on your devices, by supplying prebuilt runtimes for the common machine learning frameworks and target devices, such as the Nvidia Jetson TX2 board.
Run inference on more devices
Using integration with Amazon SageMaker and the Neo compiler, models are optimized with less than a tenth of the memory footprint so they can run on resource constrained devices such as home security cameras and actuators.
More easily run inference on connected devices
Performing inference locally on devices running AWS IoT Greengrass, reduces the latency and cost of sending device data to the cloud to make a prediction. Rather than sending all data to the cloud for performing machine learning inference, you run inference directly on the device.
Build more accurate models
Using AWS IoT Greengrass, you can perform inference and capture the results, detect outliers, and send data back to the cloud and Amazon SageMaker where it can be reclassified, tagged, and used for model retraining to improve the accuracy of your machine learning model.
Use cases
Featured customers
AWS IoT Greengrass helps Yanmar increase the intelligence of greenhouse operations by automatically detecting and recognizing the main growth stages of vegetables to grow more crops.
The Electronic Caregiver ensures high-quality caregiving with AWS IoT Greengrass ML Inference and can push machine learning models directly to edge devices and keep patients safer.
With AWS IoT Greengrass, Vantage Power pushes machine learning models to individual vehicles and detects battery faults 1 month earlier.