Reinforcement Learning (RL) is an advanced machine learning (ML) technique that learns very complex behaviors without requiring any labeled training data, and can make short term decisions while optimizing for a longer term goal.

You can use the AWS RoboMaker sample application to generate simulated training data used for RL. The RL model will teach the robot to track and follow an object. This is a simple demonstration that can be extended into use cases like worker assistance in a warehouse or an entertainment robot following a consumer in their home.

In this project, you will go through the steps to build a robotics application. This application will use reinforcement learning to train a robot (TurtleBot 3 Waffle Pi is used as an example) to drive autonomously towards a stationary robot (TurtleBot 3 Burger is used as an example). You will learn to train and evaluate the reinforcement learning model in AWS RoboMaker simulation and deploy the model to a physical robot using AWS RoboMaker fleet management.

The learning path consists of the following steps:

  • Creating an S3 Bucket, IAM Role, and Policy
  • Setting up an AWS RoboMaker development environment using AWS Cloud9
  • Using AWS RoboMaker simulation to train the reinforcement learning model and visualize the application
  • Evaluating the model through simulation
  • Deploying the model to the robot

AWS Experience: Intermediate

Time to complete: 10 hours (8 hours of model training time)

Cost to complete: This tutorial will cost you around $35*

Technologies used:

• Active AWS Account**
AWS RoboMaker
• Amazon Simple Storage Service (Amazon S3)
AWS IAM
AWS Cloud9
Amazon CloudWatch
AWS IoT Greengrass

Optional: TurtleBot 3 Waffle Pi with a camera and TurtleBot 3 Burger

*This estimate assumes you follow the recommended configurations throughout the tutorial.

**Accounts that have been created within the last 24 hours might not yet have access to the resources required for this project.

  • Step 1. Select a Region

    This robot application can be deployed in any AWS Region that supports the AWS RoboMaker service.

    You can refer to the Region table to see which Regions have the supported services. Select your Region from the dropdown in the upper right corner of the AWS Management Console.

    Region drop down

    (Click to enlarge)

    Region drop down
  • Step 2. Create an S3 bucket

    Go to the Amazon S3 console and create a bucket, this bucket is needed to store the trained reinforcement learning model. It is also used to store simulation logs. Give the bucket a name, for example, “robomaker-objecttracker-yourinitials” and select the same region selected in step 1. Keep everything else default and create the bucket.
    Bucket name

    (Click to enlarge)

    Bucket name
  • Step 3. Create an IAM role and policy

    In this step, you will create a role to allow AWS RoboMaker to run simulations on your behalf. Some actions the role performs during this tutorial include: Downloading bundled archives from Amazon S3 to create a simulation job, uploading trained reinforcement learning models to Amazon S3, and publishing metrics and ROS logs to Amazon CloudWatch during simulation.


    a. Go to the IAM console, select “Policies” on the left panel, then choose “Create policy”.

    b. In the “Create Policy” page, select “JSON” table, copy and paste the following policy. Make sure to change the ARN to your bucket then choose “Review Policy”.

    {
         "Version": "2012-10-17",
         "Statement": [
             {
                 "Effect": "Allow",
                 "Action": [
                     "cloudwatch:PutMetricData",
                     "ec2:*",
                     "iam:PassRole",
                     "logs:CreateLogGroup",
                     "logs:CreateLogStream",
                     "logs:DescribeLogStreams",
                     "logs:PutLogEvents",
                     "robomaker:*"
                 ],
                 "Resource": "*"
             },
             {
                 "Effect": "Allow",
                 "Action": "s3:*",
                 "Resource": [
                     "arn:aws:s3:::robomaker-objecttracker-yourinitials",
                     "arn:aws:s3:::robomaker-objecttracker-yourinitials/*"
                 ]
             }
         ]
     }
    c. In the “Review policy” page, give the policy a name, for example, “RoboMaker-ObjectTracker”, and choose “Create Policy”.
    Policy

    (Click to enlarge)

    Policy
    d. Go to the IAM console, select “Roles” on the left panel, then choose “Create role”.
    Role

    (Click to enlarge)

    Role
    e. In the role detail page, select “RoboMaker” as the trusted entity, and on the bottom of the page select "RoboMaker - Simulation" as your use case. This allows AWS RoboMaker to use the policy you created. Then choose “Next: Permissions”.
    StepE

    (Click to enlarge)

    StepE
    f. In the search box, search for the “RoboMaker-ObjectTracker” policy that you created previously. Check the box next to the policy to attach it to the role. Then choose "Next: Tags".
    StepF

    (Click to enlarge)

    StepF

    g. You may leave the tags section blank. Choose "Next: Review".

    StepG

    (Click to enlarge)

    StepG

    h. Give the role a name, for example, “RoboMaker-ObjectTracker-Role”. Finally, choose "Create role".

    StepH

    (Click to enlarge)

    StepH

    i. In the IAM console, find the role you just created and copy down the role ARN. You’ll need it later in the tutorial.

    StepI

    (Click to enlarge)

    StepI