This Guidance helps you reduce the time required to set up an in-plant computer vision (CV) quality detection system. It provides workflows and interfaces that enable quality subject matter experts to train CV models so that you don’t have to procure expensive data scientists. The architecture includes a series of interfaces for one-touch gateway onboarding, which eliminates the need for specialized personnel to set up, deploy, and manage the gateways and CV models. The architecture also uses edge technology, machine learning (ML), and user interfaces to initiate quality inference, orchestrate gateway deployments, calculate key performance indicators (KPIs), and visualize data through dashboards.
Architecture Diagram
Step 1
An industrial camera is used to capture product images which are stored on the edge device for the CV use case.
Step 2
The captured images are routed through the Dispatcher to the appropriate Compiled Quality Model for inference.
Step 3
The result of the inference is stored on the gateway with AWS IoT Greengrass Stream Manager and combined with other near real-time data and calculations to generate overall equipment effectiveness (OEE) and other operational metrics and identify quality issues. This is available to view in the factory through the Visualization layer.
Step 4
Near real-time data is collected from the machinery, typically from an Industrial Programable Logic Controller (PLC), using built-in connectors, like AWS IoT SiteWise Edge Data Collection Pack, or custom Protocol Adapters, and stored on the gateway.
Step 5
Operational metrics, as configured in the cloud interface, are calculated based on near real-time data, inferences, and results from the Advanced Aggregation function in the AWS IoT SiteWise Edge Data Processing Pack.
Step 6
Additional Advanced Aggregations are calculated based on Operational Technology (OT) system configurations and shift schedules and stored on the edge device. The PostgreSQL Container will manage this data on the edge.
Step 7
Data and aggregations, as configured, are transferred from the edge device to AWS IoT SiteWise for enterprise-level views and additional data analysis. AWS IoT Core handles data requiring immediate actions.
Advanced calculations, advanced alarming, and other backend services are deployed as containers in Amazon Elastic Kubernetes Services (Amazon EKS). Amazon Relational Database Service (Amazon RDS) stores event, operational, and configuration data.
Step 8
The container-based business logic and configuration data can be replicated to the edge environment and orchestrated through AWS IoT Greengrass.
Supporting features include:
a) S3 Component Sync to synchronize data between the edge device and cloud, including stored images when model re-training is requested.
b) Health Monitor to surface issues on the gateway, such as connectivity and downloaded modules.
c) Command Module to orchestrate actions between the cloud and edge device, such as requests to send stored images to the cloud.
d) Secret and Shadow Manager and Amazon SageMaker Edge Manager, which are AWS-provided components.
Step 9
Even if cloud connectivity is lost, users can view the operational data, quality results, and custom dashboard on the edge through features such as Asset Nav, an Operational Dashboard, and KPIs and Models.
Step 10
The simplified user interface (UI) helps train ML models, manage gateway deployments and enterprise operational performance, and reduce the complexity and personas required to provision a gateway in the factory. Plant managers can use the UI to monitor KPIs in their own plants or enable an organization-wide view of metrics.
Process managers can explore quality KPIs and issues on a per-line basis and examine the results of a particular part. IT engineers can train models and associate the models to particular manufacturing lines or gateways. Amazon SageMaker Edge Manager facilitates the model deployment to the specific gateway.
Step 11
Process or quality engineers train new models by requesting images from the edge device through the S3 Component Sync (custom AWS IoT Greengrass component), and storing the images in Amazon Simple Storage Service (Amazon S3).
An intuitive interface, powered by Amazon SageMaker Ground Truth, is used to identify good or bad part images and annotate defects.
Step 12
Amazon EventBridge, AWS Lambda, and Amazon Simple Queue Service (Amazon SQS) orchestrate a series of processes executed in Amazon EKS to automatically train the CV model with Amazon SageMaker and compile and package the model for the edge with Amazon SageMaker Neo.
The model is version-controlled and available for deployment to the gateway. Based on the customer use case, Amazon Lookout for Vision can be used for model training.
Step 1
An industrial camera is used to capture product images which are stored on the edge device for the CV use case.
Step 2
The captured images are routed through the Dispatcher to the appropriate Compiled Quality Model for inference.
Step 3
The result of the inference is stored on the gateway with AWS IoT Greengrass Stream Manager and combined with other near real-time data and calculations to generate overall equipment effectiveness (OEE) and other operational metrics and identify quality issues. This is available to view in the factory through the Visualization layer.
Step 4
Near real-time data is collected from the machinery, typically from an Industrial Programable Logic Controller (PLC), using built-in connectors, like AWS IoT SiteWise Edge Data Collection Pack, or custom Protocol Adapters, and stored on the gateway.
Step 5
Operational metrics, as configured in the cloud interface, are calculated based on near real-time data, inferences, and results from the Advanced Aggregation function in the AWS IoT SiteWise Edge Data Processing Pack.
Step 6
Additional Advanced Aggregations are calculated based on Operational Technology (OT) system configurations and shift schedules and stored on the edge device. The PostgreSQL Container will manage this data on the edge.
Step 7
Data and aggregations, as configured, are transferred from the edge device to AWS IoT SiteWise for enterprise-level views and additional data analysis. AWS IoT Core handles data requiring immediate actions.
Advanced calculations, advanced alarming, and other backend services are deployed as containers in Amazon Elastic Kubernetes Services (Amazon EKS). Amazon Relational Database Service (Amazon RDS) stores event, operational, and configuration data.
Step 8
The container-based business logic and configuration data can be replicated to the edge environment and orchestrated through AWS IoT Greengrass.
Supporting features include:
a) S3 Component Sync to synchronize data between the edge device and cloud, including stored images when model re-training is requested.
b) Health Monitor to surface issues on the gateway, such as connectivity and downloaded modules.
c) Command Module to orchestrate actions between the cloud and edge device, such as requests to send stored images to the cloud.
d) Secret and Shadow Manager and Amazon SageMaker Edge Manager, which are AWS-provided components.
Step 9
Even if cloud connectivity is lost, users can view the operational data, quality results, and custom dashboard on the edge through features such as Asset Nav, an Operational Dashboard, and KPIs and Models.
Step 10
The simplified user interface (UI) helps train ML models, manage gateway deployments and enterprise operational performance, and reduce the complexity and personas required to provision a gateway in the factory. Plant managers can use the UI to monitor KPIs in their own plants or enable an organization-wide view of metrics.
Process managers can explore quality KPIs and issues on a per-line basis and examine the results of a particular part. IT engineers can train models and associate the models to particular manufacturing lines or gateways. Amazon SageMaker Edge Manager facilitates the model deployment to the specific gateway.
Step 11
Process or quality engineers train new models by requesting images from the edge device through the S3 Component Sync (custom AWS IoT Greengrass component), and storing the images in Amazon Simple Storage Service (Amazon S3).
An intuitive interface, powered by Amazon SageMaker Ground Truth, is used to identify good or bad part images and annotate defects.
Step 12
Amazon EventBridge, AWS Lambda, and Amazon Simple Queue Service (Amazon SQS) orchestrate a series of processes executed in Amazon EKS to automatically train the CV model with Amazon SageMaker and compile and package the model for the edge with Amazon SageMaker Neo.
The model is version-controlled and available for deployment to the gateway. Based on the customer use case, Amazon Lookout for Vision can be used for model training.
Well-Architected Pillars
The AWS Well-Architected Framework helps you understand the pros and cons of the decisions you make when building systems in the cloud. The six pillars of the Framework allow you to learn architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable systems. Using the AWS Well-Architected Tool, available at no charge in the AWS Management Console, you can review your workloads against these best practices by answering a set of questions for each pillar.
The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.
-
Operational ExcellenceRead the Operational Excellence whitepaper
KPI and summary dashboards are a feature of this Guidance. The dashboard displays metrics such as OEE and the number of good and bad parts from the images. You can use this dashboard to examine your manufacturing operations and determine whether you are achieving expected outcomes.
-
SecurityRead the Security whitepaper
The edge-to-cloud communications use security mechanisms provided by AWS, including X.509 certificates for mutual transport layer security (mTLS) and bi-directional encryption of traffic. AWS Identity and Access Management (IAM) roles are used for deployment, and interservice communications are configured following the principle of least privilege access.
-
ReliabilityRead the Reliability whitepaper
This Guidance uses managed services, which are highly available by design. The in-plant gateway is not highly available, but the Gateway Management feature provides one-touch gateway onboarding, enabling a low recovery time objective (RTO). You can configure what data to back up and where to store that data.
-
Performance EfficiencyRead the Performance Efficiency whitepaper
Services in this Guidance were selected to reduce the undifferentiated heavy lifting for the end user while operating the solution. For example, AWS IoT SiteWise is used to collect, store, and process incoming data in near real time. This allows you to focus on KPIs important to your business rather than the underlying technology of ingesting data. AWS IoT SiteWise Edge enables the same business logic that operates in the cloud to be deployed to the edge device. This reduces efforts to maintain multiple disparate systems.
-
Cost OptimizationRead the Cost Optimization whitepaper
When choosing services for this Guidance, we considered the overall costs of maintaining the architecture, not just the costs for operating a service. For example, although AWS IoT SiteWise Edge has a significant monthly cost, this service will help you save on costs related to customer development, support, and maintenance.
-
SustainabilityRead the Sustainability whitepaper
This Guidance uses a combination of managed services and services that elastically scale based on demand. As managed services, AWS IoT Core, AWS IoT SiteWise, and Amazon S3 provide compute and storage on demand. Amazon EKS is configured to scale up and down based on usage.
Implementation Resources
A detailed guide is provided to experiment and use within your AWS account. Each stage of building the Guidance, including deployment, usage, and cleanup, is examined to prepare it for deployment.
The sample code is a starting point. It is industry validated, prescriptive but not definitive, and a peek under the hood to help you begin.
Related Content
[Title]
Disclaimer
The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.
References to third-party services or organizations in this Guidance do not imply an endorsement, sponsorship, or affiliation between Amazon or AWS and the third party. Guidance from AWS is a technical starting point, and you can customize your integration with third-party services when you deploy the architecture.