As computing power extends into Internet of Thing (IoT) devices, software becomes more and more crucial to making decisions, processing data, and providing insight to end users at the device. Devices are the backbone of many businesses and applications. Regardless of the use case, customers benefit from quick, widespread deployment of software updates and improvements across device fleets. With AWS IoT Greengrass V2, AWS customers can easily build, deploy, and manage custom device software as AWS IoT Greengrass components. AWS IoT Greengrass handles activities such as tracking the component versions, managing the fleets of devices, and orchestrating component updates.

In this blog, we will demonstrate how to automate the deployment of component changes on AWS IoT Greengrass. This solution reduces the time it takes developers to deploy custom device software across a fleet from minutes to seconds, saving developers valuable time and improving agility.

Introduction

This solution uses a centralized code repository with Continuous Integration and Continuous Deployment (CI/CD) to assist in following DevOps best practices. For more information about CI/CD and DevOps on AWS, reference the Practicing Continuous Integration and Continuous Delivery on AWS whitepaper.

In this article, we will present a method for automating the deployment of custom AWS IoT Greengrass components. AWS IoT Greengrass includes AWS-provided components to add common functionality to your devices and also allows for custom components creation. Custom components may analyze data, display a front-end dashboard, or run an application with intermittent connectivity to the cloud. For demonstration purposes, we will be using three pre-built components. However, this solution is easily adaptable to components you may already have running in your environment.

This solution clones a Github repository to an AWS CodeCommit repository as an example implementation. Other implementations can be used as long as the code is deployed to CodeCommit. At the beginning of the steps below, we will outline how to clone a repository from Github into AWS CodeCommit. However, other code repositories can be imported into AWS CodeCommit if necessary for your use case. For more information on migrating code to an AWS CodeCommit repo, see the documentation here.

Prerequisites

For this walk through, you should have the following prerequisites:

• An AWS account
• A basic understanding of:
  • AWS CloudFormation
  • AWS IoT Core
  • AWS IoT Greengrass
  • AWS CodeBuild
  • AWS CodeCommit
  • AWS CodePipeline

Install AWS CLI

Make sure you have installed and setup AWS CLI with the necessary permissions:

• AWS CLI Setup

Install CDK

Here is a guide to Get Started with AWS CDK:

• Installing aws-cdk

Please verify the cdk package version installed is 2.x or higher.

Solution Overview

The solution utilizes an AWS CodeCommit repo to store the component code, a Lambda function to trigger the build, and AWS CodeBuild to orchestrate the deployment of the updated component. Finally, AWS IoT Greengrass pushes the updated component out to the devices.

• A commit is made to the AWS CodeCommit repository. An Amazon CloudWatch Event event has been configured such that any time a commit is made to the configured repository, the event occurs.
• An AWS Lambda function is triggered by the Amazon CloudWatch Event.  The AWS Lambda function first determines if the commit was made on a file that is part of the source code for the component. If that is the case, it saves the component name(s) as AWS CodeBuild Project Environment Variables and starts the AWS CodePipeline.
• AWS CodeBuild job is triggered by the AWS CodePipeline. The AWS CodeBuild job runs a shell script which deploys the component to the devices using the AWS IoT Greengrass API.

AWS Cloud Development Kit (AWS CDK) is a framework for defining cloud infrastructure in code, and provisioning it via AWS CloudFormation. If you are new to the AWS CDK, follow the getting started guide. 

The CDK will deploy the following resources in the AWS Account:

• AWS CodePipeline
• AWS Lambda
• CodeBuild
• Amazon S3
• [Optional] Amazon EC2
• AWS CLI v2
• AWS CDK v2

Deploy the solution

Create and clone code repository

• Create CodeCommit Project
  • Go to AWS CodeCommit Console
  • Select Create Repository
  • Provide name for the repository. For example ggv2-cdk-blog-test
  • Choose Create
• Clone the CodeCommit repository on your local machine, for example if the CodeCommit repository is named  ggv2-cdk-blog:

  • git clone codecommit::us-east-2://ggv2-cdk-blog
    cd ggv2-cdk-blog

• In order to deploy the cdk you will need to copy the cdk contents from github repository:
  • To easily copy the contents of this github project to your new project, copy export.zip to your CodeCommit project directory, and unzip
    • Note: The .gitignore file is part of export.zip, if you don’t find the file after unzip check your settings to view hidden files in the IDE

Source code updates

Please update following attributes in cdk.json file with appropriate values:

• account
  • Account ID of your AWS account, for example: 1234567890
• codecommit_repository_arn
  • For example: arn:aws:codecommit:us-east-1:111111111111:MyDemo*
• region
  • For example: us-east-1
• create_core_device
  • acceptable values are true or false
  • Even if you choose false make sure to provide a name for an existing core device in the option core_device_name and core_device_group_name
• default_branch_name
  •  branch to track for the CodePipeline. For example: main
•   core_device_name
  •  The name of your new/existing Greengrass core device.
•  core_device_group_name
  • The name of your new/existing Greengrass core device group.
•  project_prefix
  •  Any string value

Deploy CDK pipeline

• Set up your virtualenv for Python. You may need to use python3 in replacement of python, depending on your local python configuration.

  • python3 -m venv .venv
    source .venv/bin/activate
    python3 -m pip install -r requirements.txt

• Bootstrap your account/region for CDK – replace the appropriate variables (i.e. ACCOUNT-ID, REGION, ADMIN-PROFILE) before executing.
  • Commands to run:

    • export CDK_NEW_BOOTSTRAP=1 
      npx cdk bootstrap --cloudformation-execution-policies arn:aws:iam::aws:policy/AdministratorAccess \
      aws://ACCOUNT-ID/REGION 

    •  Note: Verify you have the AdministratorAccess policy in your AWS account or you can customize the policy to be used by AWS CDK in order to create AWS resources
    • For example, like this:

    • export CDK_NEW_BOOTSTRAP=1                                    
      npx cdk bootstrap --cloudformation-execution-policies arn:aws:iam::aws:policy/<CustomPolicy> \
      aws://123456789/us-east-2

• Commit updates to the repository and deploy the CDK app. You may need to git push origin <main branch name> , instead of git push

  • git add --all
    git commit -m "initial commit"
    git push
    cdk deploy

•  Optional: To update export.zip in your own project, run the following:

  • git archive -o ./export.zip HEAD

The code repository for this blog has sample custom AWS IoT Greengrass components, that will display the message Hello World in the log file of the component. The next section will provide more information about building your own AWS IoT Greengrass components.

Adding your own AWS IoT Greengrass components

The code in this blog uses Greengrass Development Kit (gdk cli) in order to build and publish Greengrass components. For more information please check this documentation.

To add new components to the project, create a new component directory in the components directory. Make sure your components include the following:

• gdk-config.json (GDK configuration file)
• buildspec.yml (for CodeBuild)
• requirements.txt (for Python dependencies; currently used by provided buildspec.yml examples)

Building AWS IoT Greengrass components

Here are 5 tips to build AWS IoT Greengrass v2 Components. For more information please refer below:

• Develop AWS IoT Greengrass components
• Use the AWS IoT Device SDK to communicate with the Greengrass nucleus, other components, and AWS IoT Core

Cleaning Up

• CDK Cleanup
  • Run the following command from your terminal on the path where the code repository exists (Example: Users/johndoe/desktop/ggv2-cdk-blog ~ %)
  • cdk destroy

Conclusion

You now have setup a DevOps pipeline for multiple components in the same code repository. With these enhancements, when a developer pushes code updates to components in the repository, the AWS IoT Greengrass V2 components automatically deploy a new version with the updates. Expect to see developers be able to iterate faster, creating business value and innovation at a new pace.

Learn more about AWS IoT Greengrass through this hands on workshop.  For more information about public components for AWS IoT Greengrass, please visit https://github.com/aws-greengrass

About the Authors

Jon Slominski is a Sr. Solutions Architect with the Prototyping & Cloud Engineering (PACE) team at AWS. Building prototypes focused on IoT, AI/ML, and robotics, Jon helps customers innovate and envision the art of the possible. Outside of work, Jon enjoys spending time and traveling with his wife and daughters.

Joyson Neville Lewis is an IoT Data Architect at AWS Professional Services. He has worked as a Software/Data engineer before diving into the Conversational AI and Industrial IoT space where he works with companies to connect the dots between business and AI using Voice Assistant/Chatbot and IoT solutions.

Jack Tanny is an Associate Data/ML Engineer in AWS’s Professional Services team. He builds solutions that use data to solve problems and unlock business value for our customers. In his free time, you can usually find Jack in the mountains, biking, skiing, or camping.