Implementing sensor workflows using AWS Snowcone and AWS IoT Greengrass

In our first blog of this series, we covered using an IoT device to store the sensor data on an Amazon EC2 instance running on AWS Snowcone. That use case covered collecting data from sensors in locations such as a factory floor or a mine with austere network connectivity.

There are other instances where you use a sensor for predictive maintenance such as in the case with farm or industrial equipment. You can use the sensor to detect when supplies are low or components are due for replacement. The sensor can trigger a notification that you can use for scheduling maintenance.

In this blog, we explore how to send the sensor data to the cloud and perform event-driven workflows as mentioned using AWS IoT Greengrass. We also explore how the incoming data can be persistently stored on a local AWS Snowcone device or used for running inference. For the use case in this post, you need connectivity to the cloud.

About AWS IoT Greengrass

AWS IoT Greengrass is an IoT service that provides complete management, security, and monitoring for your sensor network. AWS IoT Greengrass provides compute, messaging, and syncing services from the edge to the AWS Cloud, and AWS IoT Greengrass does not require constant, high-bandwidth connectivity between the two. AWS IoT Greengrass is able to switch to a disconnected/shadow mode when the network is offline; however, connectivity is a necessity for centralized management.

Sending an Amazon SNS Notification using the sensor

The following architecture diagram illustrates the on-premises sensor with AWS IoT. The ESP32 sensor connects to the internet through a wireless router. The communication is secured using the AWS IoT framework.

First, we configure the ESP32-based sensor button to work with the AWS IoT framework. This includes creating the necessary roles, certificates, and enabling access to AWS IoT Core.

1. Obtain your AWS credentials from the AWS Management Console, and create your default AWS profile. You need the access key and secret key for the user you use. This must be done on the workstation where you are running Mongoose OS (mos).
2. Once you have configured your AWS profile, run the following command in the mos window.

> mos aws-iot-setup --aws-region REGION --aws-iot-policy mos-default

3. Log in to the AWS Management Console, and check if the device is communicating with the cloud (AWS IoT). Click on Things on the left navigation pane. In the resulting listing, you will see your IoT sensor button listed as exp32_some-suffix.

4. Select your device to see the details, including the Amazon Resource Name (ARN).

5. In your AWS Management Console, click on the Test link in the left navigation pane, then enter the subscribe topic as “mos/topic1”.

6. Next, physically press the ESP32 button. You will receive the button press event.

Set up an Amazon SNS notification

In this section, you make the button press event trigger an Amazon SNS notification to the end user.

1. In the AWS IoT console, in the left navigation pane, click on Rules under the Act section:

2. Click on the Create a rule button, and enter the Rule query statement as SELECT * from ‘mos/topic1’ where buttonStatus=’1’.

3. Click on Actions, where you are able to select from a number of different actions, select the send an Amazon SNS message radio button.

4. Configure the SNS target as “button_pressed”.

5. From the AWS Management Console, select Amazon SNS. Next, select the topic, and add a subscription with an email ID.

6. You’ll receive an email notification asking you to subscribe to the topic. Subscribe to the topic.
7. Press the button. You will receive an email notification with the event details, as shown in the following screenshot.

Simplify sensor workflow with AWS IoT Greengrass and AWS Lambda

In the previous section, we used AWS IoT Greengrass to trigger a notification when the IoT button is pressed. Now, we explore how to run a Lambda function that persists data on Snowcone. We save the data to the file system on an EC2 instance running on Snowcone. We use AWS IoT Greengrass to manage the orchestration of the entire scenario.

1. From the AWS Console/IoT service, create a new AWS IoT Greengrass group and name it. In our example, we typed in “ButtonPress”.

2. Next, create a Greengrass Core.

3. In the next screen, you can see the code and certificates to install AWS IoT Greengrass on the device. You use an EC2 instance running on Snowcone to install AWS IoT Greengrass.

Download the software, certificates, and configuration files from the links as shown in the following screenshot:

4. Click on the appropriate links to download the resources as tar.gz.
5. Click on Choose your platform to download the AWS IoT Greengrass software for Linux.
6. Get the Certificate Authority (CA) certificates. In this example, we used the RSA 2,048-bit key from Amazon Root CA and copy paste the certificate as cert.

Deploy AWS IoT Greengrass on EC2 instance

If you deploy the AWS IoT Greengrass software on the local Snowcone based EC2 instance, you can extend AWS Lambda functions to the edge. This means that based on the triggers, such as pressing the button, you can process the incoming data or store it. In this case, we store the data using a Lambda call.

1. Copy the downloaded resources from the earlier step to a temporary directory on your EC2 instance and complete the prerequisites for deploying AWS IoT Greengrass on Linux. Ensure you are running the correct kernel version, and set up an AWS IoT Greengrass user and group before proceeding to the next step. In our case, you have already completed the documented steps for Linux.
2. Extract the tar file for the Greengrass core software:

$ tar -xvf /tmp/greengrass-linux-x86-64-1.11.0.tar.gz -C /

3. Unpack the certificate bundle to the certs directory and config.json to the config directory:

$ sudo tar -xvf /tmp/14bbe8310a-setup.tar.gz -C /greengrass

4. Copy the root CA certificate to the certs directory:

$ sudo cp AmazonRootCA.cert /greengrass/certs

5. Edit and change the caPath to the CA root certificate you saved earlier:

$ sudo vi /greengrass/config/config.json  


---- lines removed --- 

"caPath" : "file:///greengrass/certs/AmazonRootCA.cert"
  }

6. Create a directory to save your incoming messages (or sensor data). In our example, you will create a directory /sensordata, which is owned by the AWS IoT Greengrass user.

$ sudo mkdir /sensordata 
$
$ sudo chown ggc_user: ggc_group /sensordata

7. Start the greengrassd daemon:

$ sudo /greengrass/ggc/core/greengrassd start
Setting up greengrass daemon
Validating hardlink/softlink protection
Waiting for up to 1m10s for Daemon to start

Greengrass successfully started with PID: 6373
$

Check your log file /greengrass/ggc/var/log/system/runtime.log for any startup or runtime issues. This log is critical in troubleshooting configuration issues.

Configure AWS IoT Greengrass

In this section, we add the sensor to the IoT framework and then create an AWS Lambda function.

1. Log in to your AWS Management Console, and add the IoT device (ESP32 sensor). Click on the Select an IoT Thing button:

2. Click on the radio button corresponding to the device, and then click Finish.

Next, create a simple function to trigger a Lambda function when the button is pressed. Then, create a simple function to trigger a Lambda function when the button is pressed. You see that AWS IoT events are passed to the AWS Lambda function. The message contains the “eventTime” and any variables received from the IoT. In the next step, add a new Lambda function.

3. Go to your AWS Management Console, select Lambda → Create function → Author from Scratch. Then, enter the function name as “SaveButtonStatus”, and select the Runtime as Python 2.7. Afterward, click the Create Function button. In the next screen, enter the following function (code):

def lambda_handler(event, context): 
 import json
 from datetime import datetime    
 # Example: 2020-08-25T19:50:53.653555
 now = str(datetime.today().isoformat())
 serial = (event['iotSerial']) 
 status = (event['buttonStatus']) 
  
# JSON Data to be written 
 contents ={
  "iotSerial" : serial,       
  "buttonStatus" : status,
  "timestamp" : now 
 } 
  
# Serializing json  
 json_object = json.dumps(contents, indent = 4 ) 
  
# Writing to sample.json 
 filename = ("/sensordata/" + serial + "-" + now + ".json")
 print (filename)
 with open(filename, "w") as outfile:
  outfile.write(json_object)
 return 0

You trigger this code when the IoT button is pressed.

4. From the Lambda console, test the code. Configure a test event, and once your tests have passed, publish the new function as a new version.

Add the Lambda function to the AWS IoT Greengrass group

In the following section, you deploy the Lambda function that you created earlier to the AWS IoT Core.

1. In the AWS Lambda console, select AWS IoT, click on AWS IoT Greengrass, and then Groups.

2. In the left navigation pane, click on Lambdas → Add Lambdas, and add your AWS Lambda function to the AWS IoT Core.

3. From the Group view, click on Subscriptions. In the Select a source pulldown, add IoT Cloud, and in the Select a target, click on Lambdas, and select the SaveButtonStatus function, which we published earlier:

4. In the next screen, enter the topic as “mos/topic1”.

5. After saving your configuration, you can see all the subscriptions you configured for the AWS IoT Greengrass group called ButtonPress.

6. The Lambda function must save the incoming sensor payload to the Snowcone’s EC2 instance’s local file system /sensordata . By default, Lambda functions have read-only access to the disk. To configure access, click on Groups, and then Resources. Provide a name, select Volume, and provide the path as /sensordata for source and destination paths. Then, in the Access permissions, select Automatically add OS group permissions of the Linux, and select the Lambda function that you want writing to the volume.

Now, you have added a data volume on the Amazon EC2 instance that the AWS Lambda function uses to write files:

7. From the Actions menu, select Deploy. This pushes the function and changes to the EC2 instance running AWS IoT Greengrass. The status will say Successfully completed.

Verify the workflow

In this section, you verify the end-to-end solution. You receive a notification and also see the data is persisted on the EBS volume attached to the Snowcone-based EC2 instance.

1. Press the IoT button. You receive an email notification with the event details.
2. Log in to your Amazon EC2 instance.
3. List files on your sensor data directory /snowcone on the EC2 instance to verify that the message has been saved. You will see a file with the timestamp matching the button press event.

$ sudo ls -l /sensordata/
total 4
-rw-r--r--. 1 ggc_user ggc_group 107 Sep 25 17:52 AC3D45C667-2020-09-25T17:52:10.702510.json
$

4. View the contents of the JSON data file:

$ sudo cat /sensordata/AC3D45C667-2020-09-25T17\:52\:10.702510.json 
{
    "timestamp": "2020-09-25T17:52:10.702510", 
    "iotSerial": "AC3D45C667", 
    "buttonStatus": "1"
}
$

Cleaning up

When you are done with the resources you created using AWS IoT Core and AWS Lambda, you may want to clean them up to avoid incurring unwanted future charges. Remember to delete unwanted resources and disable permissions that are no longer required. For the EC2 instance on your AWS Snowcone, consider terminating the EC2 instance to free up CPU and memory resources on the the device. If you are done working with the AWS Snowcone, you should return it to AWS and avoid incurring additional extra-day fees. To return the AWS Snowcone, disconnect it from your USB-C power supply, and the E Ink display will change to show the courier shipment return label. You can contact the courier to pick up the AWS Snowcone, or drop it off at a courier drop-off location.

Summary

In this second blog of our two-part series, we simplified the IoT-based sensor network workflow with AWS IoT Greengrass, and we also secured the entire application using AWS services. The management, build, and deployment of your software is all performed using AWS IoT Greengrass.

We demonstrated how AWS IoT Greengrass, AWS Lambda, Amazon EC2, and Amazon EBS can be leveraged for implementing a sensor triggered workflow that you can use for event-driven predictive maintenance. This automation improves your operational efficiency. We used ESP32 and Mongoose OS to build a simple sensor workflow. Then, we showed you how to use AWS IoT Greengrass to secure and simplify the complex DIY solution. As you can see, AWS Snowcone can be leveraged to build your IoT applications at the edge while providing you with a compute and persistent storage solution.

We’re interested in hearing what you think about this blog post. Please share your thoughts and questions in the comments section.