Introduction

Many modern industrial operations require extensive monitoring and real-time decision making for efficiency and safety reasons. To reduce the unexpected network interruption and delay in IoT data processing, edge computing becomes a desirable option for real-time IoT data processing and monitoring. Edge computing is a system of micro computing/storage devices that are installed at the edge of the network to efficiently process data locally. For industrial sites, such as remote power plants, oil rigs, and factory floors, an edge application with a user interface (UI) component offers great advantages to automate localized operations and improve workforce experience.

However, traditional IoT applications at the edge can experience some challenges, such as lack of centralized life-cycle management and a long development cycle. AWS IoT Greengrass V2, the latest open source edge run time from AWS, offers a pathway to address the above-mentioned challenges. With AWS IoT Greengrass, you can access pre-built software components to improve the modularity of your application and accelerate your development efforts. AWS IoT Greengrass also offers centralized security measures through a built-in log manager to facilitate edge component deployment. Moreover, the integration of AWS IoT Greengrass with AWS IoT Device Management enables you to monitor and manage your device fleet at scale.

Since AWS IoT Greengrass V2 launched, AWS is actively helping AWS Partner, TensorIoT, to develop various functional applications at the edge using this service. Such applications cover use cases such as: connection to industrial systems and IoT devices, low latency data processing and analytics to derive operational insights, and artificial intelligence (AI)/machine learning (ML) applications at the edge. In this blog, we will showcase a UI application at the edge with AWS IoT Greengrass V2, a jointly developed solution by TensorIoT and AWS Professional Services. This UI application at the edge contains multiple custom AWS IoT Greengrass components to achieve flexible data ingestion, and IoT data analytics and visualization at the edge. The application deployment and updates on multiple edge devices can be efficiently implemented via AWS IoT Greengrass runtime.

In this blog, we will showcase how to use this UI at the edge to address the following remote industrial job monitoring use cases:

Use case 1 (part 1 of the blog)

Operators on factory floors may have some reports or IoT job metadata that cannot be ingested via a pre-defined IoT data model to the cloud. This UI provides a file upload page, that uses a front-end AWS IoT Greengrass V2 component to ingest a JSON file, and sends the file data to a back-end component. Then the back-end component will send the data to an AWS IoT Greengrass V2 pre-built component: stream manager. The stream manager component makes it easier and more reliable to transfer high-volumes IoT data from the edge to the cloud. As a pre-built component, it offers flexible target destinations in the cloud, including Amazon Simple Storage Service (Amazon S3), Amazon Kinesis, AWS IoT SiteWise and AWS IoT Analytics. It is designed to work in environments with intermittent or limited connectivity. You can define multiple data exportation configurations when the edge device is connected or disconnected. You can also define bandwidth use, timeout behavior, and set priorities, so that the critical data can be exported first when internet connection resumes.

This stream manager component will achieve two goals:

• local data storage;
• sending data from the local edge device to AWS.

In this use case, the unmapped data will land in Amazon S3. A variety of data analytics tools (Amazon Glue and Amazon Athena) /BI tools can be used to analyze this metadata with other IoT time series data in a flexible manner.

Use case 2 (part 2 of the blog)

Operators need to make judgements with streaming IoT data in a near-real time fashion. The data will help operators make the right decisions if it can be shown in an interactive UI application with low latency. In part 2 of this blog, you will learn how to build a UI application at the edge to show streaming IoT job data.

The images below highlight the UI application developed by AWS Professional Services and TensorIoT. You can use this app to remotely monitor IoT job status in near-real time.

Figure 1: UI application at the edge developed by AWS Professional Services and TensorIoT for remote IoT job monitoring in near-real time.

The following walk through contains detailed steps to develop this UI application at the edge with AWS IoT Greengrass V2. To build a UI solution based on the instructions shared in this blog, you don’t need specialized data analytics or IoT data processing experience.

Solution overview

Here is the solution architecture of this workflow at the edge. Four user-defined components will be built in this two-part blog:

• A front-end AWS IoT Greengrass UI component written in ReactJS, which contains all UI pages used in both use cases (part 1 of blog);
• A JSON file upload component written in Python, that collects file data from the front-end UI, and sends the JSON information to Amazon S3 via stream manager (part 1of blog);
• A dummy publisher publishes wind turbine data at a rate of one message every 10 seconds, and sends the message to a WebSocket component via IPC protocol (part 2 of blog);
• A WebSocket component serves the IoT messages from the dummy publisher component to the UI component via a WebSocket server (part 2 blog).

Besides these custom components, there are two pre-built AWS IoT Greengrass V2 components deployed with this solution:

• Log manager for application monitoring
• Stream manager for edge device data exportation

In this blog, an Ubuntu 22.04 LTS EC2 instance with AWS IoT Greengrass v2 runtime is used to simulate an edge device with AWS IoT.

Figure 2: The overall diagram of the UI application at the edge.  

Prerequisites

For this walk through, you should have the following prerequisites:

• An AWS account. If you don’t have an AWS Account, follow the instructions to create one, unless you have been provided event engine details.
• A user role with administrator access (service access associated with this role can be constrained further when the workflow goes to production).
• Recent modern browser (Latest version of Firefox or Chrome)
• No specialized knowledge is required to build this solution, but basic Linux and Python knowledge will help.

Walk Through

This two-part post contains the following walk through steps and will provide detailed instructions for developing this solution:
Part 1 for setting up runtime and building a JSON file upload application (this post):

• Step 1: Setup Amazon S3 bucket
• Step 2: Setup AWS IoT Greengrass V2 and dependencies
• Step 3: Launch a UI component
• Step 4: Launch a file upload application
• Step 5: Test the file upload application

Part 2 for building the IoT job monitoring application:

• Step 1: Launch a dummy publisher
• Step 2: Launch a WebSocket component
• Step 3: Configure deployment for log manager
• Step 4: Test IoT job monitoring UI page

Please note that this post features the key solution milestones for conciseness, but readers should visit the GitHub repository for a full walk through and source code.

Step 1: Setup an Amazon S3 bucket

First, create a bucket in Amazon S3 using the AWS Management Console. Please use these instructions to create an Amazon S3 bucket and take a note of the bucket name. You need to provide this Amazon S3 parameter to the JSON file upload component in Step 4 of this walk through.

Second, you can use similar steps to create an Amazon S3 bucket for saving edge component artifacts. Please take a note of this Amazon S3 bucket so you can use it in the component deployment steps.

Step 2: Setup AWS IoT Greengrass V2 on an Amazon Elastic Cloud Compute (Amazon EC2)  instance

1. Install dependencies on the Amazon EC2 Instance

Set up the Amazon EC2 instance of type t2.medium with ubuntu 22.04 base image in the AWS account with the following user data:

sudo su
apt-get update
apt-get install -y python3-pip zip jq
git clone https://github.com/aws/aws-iot-device-sdk-python-v2.git
python3 -m pip install ./aws-iot-device-sdk-python-v2

After the Amazon EC2 instance status changes to running, use these instructions to setup an AWS session manager access role to access this Amazon EC2 instance without SSH. Next, use Connect with the Amazon EC2 instance via AWS Systems Manager Session Manager (SSM) section of this blog to access the Amazon EC2 instance via SSM. Finally, install AWS Command Line Interface (CLI) V2 on the Amazon EC2 instance:

wget https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip
unzip awscli-exe-linux-x86_64.zip
sudo ./aws/install

2. Install AWS IoT Greengrass on the Amazon EC2 instance

You can use Install Greengrass on Amazon EC2 instance section of this blog to install AWS IoT Greengrass on the edge device.

Before you launch the following custom components, please ensure you add the necessary Amazon S3 access policy (e.g. PutObject and GetObject) to the AWS IoT Greengrass role alias so that this IoT device can access Amazon S3 according to these instructions.

Step 3: Launch the UI component

Components are building blocks that enable easy creation of complex workflows such as local data processing, messaging, data management and machine learning inference. With the latest IoT Thing Group definition from AWS IoT Greengrass V2, any updates made by UI designer can be efficiently orchestrated to all destination targets by revising the component deployment on the AWS IoT Thing group.

The following steps show how to deploy the previous packaged ReactJS UI code as an edge component on AWS Greengrass V2.
1. Change the directory to components/edgeui/aws-gg-deploy in the directory where the GIT repository was cloned to the Amazon EC2 instance.

2. Modify the deployment script deploy-edge.sh by replacing the following placeholders with your customized values in _setEnv() section:

YOUR_AWS_ACCOUNT_NUMBER
YOUR_AWS_REGION
ROLE_ARN
S3_BUCKET for edge component artifacts

Press Ctrl-X, then press Y to save the modified deploy.sh file with the same name.

3. Run the following script to deploy this component:

export AWS_ACCESS_KEY_ID=REPLACE-WITH-YOUR-AWS_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY= REPLACE-WITH-YOUR-AWS_SECRET_ACCESS_KEY
export AWS_SESSION_TOKEN= REPLACE-WITH-YOUR-AWS_SESSION_TOKEN
chmod 744 deploy-edge.sh
 ./deploy-edge.sh

This bash script takes approximately 90 seconds to finish. Once it exists, please confirm that you have the following component created in your AWS IoT Greengrass V2 console, and its latest status is shown as running:

Figure 3: UI component status shown in AWS IoT Greengrass console.

Please note this UI component contains all UI pages used in both use cases of this two-part blog.

Step 4: Launch a JSON file upload application

Next, the JSON file upload component can be launched. This component uses a Flask application to receive the JSON file data from the front UI component. Once it receives the JSON file, it will send the JSON file to Amazon S3 by using the stream manager component. The stream manager component will be launched with the file upload component by defining it as a component dependency in the recipe file as:

ComponentDependencies:
  aws.greengrass.StreamManager:
    VersionRequirement: '^2.0.0'
  aws.greengrass.TokenExchangeService:
    VersionRequirement: '^2.0.0'
    DependencyType: HARD

To launch the JSON file upload component, you can use the same step as Step 3 to modify the deployment script deploy-edge.sh  in components/com.fileUploader/aws-gg-deploy folder.

Next, modify the recipe-file-template.yaml file in the same directory by adding arguments of your S3 bucket name that was setup in Step 1 in Run command of Manifests section as:

Run: Script: python3 -u {artifacts:decompressedPath}/$artifacts_zip_file_name/$artifacts_entry_file -b="Replace with your Amazon S3 bucket name"

Please replace the default Amazon S3 bucket name with your own Amazon S3 bucket you created in step 1. Save this recipe template file with Ctrl-X, then press Y to save the modified recipe file with the same name.

Finally, run the same commands as shown in Step 3, to execute deploy-edge.sh.

This bash script takes approximately 30 seconds to finish. Once it exists, please confirm that you have the following component created in your AWS IoT Greengrass V2 console, and its latest status is shown as running:

Figure 4: Status of the JSON file upload component shown in AWS IoT Greengrass console.

Step 5: Test the JSON file upload application

Open a web browser on your laptop, and log in UI with the following address:

http://localhost:8080/fileUpload

The following file upload UI page should be shown:

Figure 5: JSON file upload UI page.

Select Upload run configuration file from the middle box of the UI and upload a JSON file of your choice. There is one sample JSON file provided in the Github repository (in the folder: jsonfile_sample)for you to upload for testing purposes, and it contains sample metadata for a wind turbine inspection job. Once you select the file, choose Start inspection to send the file data to the com.fileUploader component and the file data will be uploaded to the Amazon S3 bucket under the folder ggstreamdata as:

Figure 6: JSON file uploaded to S3 by using the Filer Uploader application.

Summary of part 1

This blog shows a UI application at the edge, jointly developed by AWS Professional Services and AWS Partner, TensorIoT, for remote industrial job monitoring and processing. In part 1 of this two-part series, you learned how to use the latest AWS IoT Greengrass V2 run time to develop a file upload application for ingesting unmapped IoT metadata. With AWS IoT Greengrass V2 run time and pre-built components, you can accelerate your development efforts for an IoT application at the edge. The modularized components developed in this application can be easily deployed and updated to multiple IoT devices.

In part 2, you will learn how to develop a second IoT application at the edge to monitor remote IoT jobs with a custom Greengrass component for processing real-time IoT data. Next, you will learn how to use a pre-built Greengrass component, log manager, for monitoring Greengrass component health. Finally, you will be able to test the functionality of the IoT application at the edge.

Call to action

Please finish the whole solution by visiting part 2 of this blog post series.

About the author

	Julia Hu is a Sr. AI/ML Solutions Architect with Amazon Web Services. She has extensive experience in IoT architecture and Applied Data Science, and is part of both the Machine Learning and IoT Technical Field Community. She works with customers, ranging from start-ups to enterprises, to develop AWSome IoT machine learning (ML) solutions, at the Edge and in the Cloud. She enjoys leveraging latest IoT and big data technology to scale up her ML solution, reduce latency, and accelerate industry adoption.
	Martin Lehofer is Practice Manager for Industrial Data with Amazon Web Services. Martin has extensive Industrial IoT experience in a wide range of verticals and applications; from Predictive Maintenance, Data Analytics, Distributed Computing to Artificial Intelligence. At AWS Professional Services, Martin leads a team of architects and engineers helping strategic industrial customers to achieve their business outcomes using data analytics.
	Joyson Neville Lewis is an IoT Data Architect at AWS Professional Services. He has worked as a Software/Data engineer before diving into the Conversational AI and Industrial IoT space where he works with companies to connect the dots between business and AI using Voice Assistant/Chatbot and IoT solutions.
	Tanya Lobo Parmar is a Director with TensorIoT, overseeing operations in the EMEA region. She is focussed on getting more customers in European markets to modernize and grow using AWS Cloud. She also enjoys managing challenging projects in IoT, Machine Learning and Managed AI.
	Vidya Ramaswamy is a Senior Software Engineer at TensorIoT, an AWS Partner and has 8+ years of experience in software development. She enjoys coding and developing various IoT solutions.
	Rajeev Pulleti is a Frontend Engineer from TensorIoT with proficient knowledge in Swift, Objective-C, and Javascript.