AWS Robotics Blog

Testing map generation at scale with 3D worlds from AWS WorldForge

In this blog, learn about AWS RoboMaker WorldForge, and how 3D world generation can be used to streamline testing and training of core robotic algorithms at scale.

Introducing AWS WorldForge:

Mapping algorithms and localization accuracy is fundamental to robotics applications, be it a cleaning robot within a home or an Autonomous Mobile Robot (AMR) in a warehouse environment. 3D Simulation of robots enables up front testing of algorithms resulting in reduced quantity, frequency and time take to fix errors discovered in the field. Real world errors and long periods of downtime create a negative end user experience for users, and can erode customer trust in the robotics company and the robot itself. Setting up the infrastructure to create 3D simulation worlds to test these algorithms is costly, complex and resource-intensive. Varieties of worlds are required to effectively test for issues seen in the real world. As a result, resources required to build these 3D simulation worlds deters robotics companies from using simulation-based testing approaches at scale.

AWS RoboMaker WorldForge makes it faster, simpler, and less expensive to generate a multitude of virtual 3D worlds with a single API call. This ability to generate worlds with an API removes the complexity and high cost in creating the vitally needed variety in the simulation worlds, and provides direct integration with AWS RoboMaker simulation jobs. We cover this in the next three sections.

In the first section, you learn how to use AWS RoboMaker WorldForge to generate 3D simulation worlds, or “WorldForge worlds”. This is done by creating a simulation world template that uses a collection of parameters to determine how to generate new worlds. We then create 3D simulation worlds from the simulation world template by calling the create world generation job API and have a group of worlds available to test our application in.

In the second section, you will set up an open-sourced sample application provided to run on AWS RobMaker. In the sample application, a virtual robot explores a WorldForge world and creates a map. The sample application uses the expore_lite exploration Robot Operating System(ROS) package with move_base and gmapping ROS packages to create a map of the WorldForge world.

In the final section, you run the sample application with multiple 3D worlds concurrently, by creating an AWS RoboMaker Batch simulation job. AWS RoboMaker imports the WorldForge worlds seamlessly into your simulation application based on the configuration provided. The generated map files from all the AWS RoboMaker simulation jobs are uploaded to a common Amazon Simple Storage Service (Amazon S3) bucket. Your Quality Assurance (QA) engineers and robotics developers can then access and determine which environments and tests are challenging for the robot, and which corresponding worlds can be used in subsequent tests to verify the improvements.

Prerequisites:

You must have an Ubuntu 18.04 machine. The machine needs the ROS Melodic environment with colcon, colcon bundle, rosws and the AWS Command Line Interface (AWS CLI) setup, along with the credentials to upload files to Amazon S3 and make AWS RoboMaker API calls.

Creating WorldForge worlds:

To create a set of WorldForge worlds, you create a simulation world template that uses a collection of parameters and components to determine how to generate the new worlds. Components include the floor plan, preferences for interior materials, furniture, assets, etc. You can learn more about Simulation World Templates here.

Save the following JSON to a file called template.json. The file contains the parameters and the components of the template that describe the world to be generated.

{
    "Version": "1",
    "Buildings": [
      {
        "Floors": [
          {
            "Floorplan": {
              "Ceiling": {
                "Height": 3
              },
              "DesiredConnections": [
                {
                  "ConnectionType": "Doorway",
                  "Location": [
                    "Bathroom",
                    "Livingroom"
                  ]
                },
                {
                  "ConnectionType": "Opening",
                  "Location": [
                    "Livingroom",
                    "Kitchen"
                  ]
                },
                {
                  "ConnectionType": "Doorway",
                  "Location": [
                    "Bedroom",
                    "Livingroom"
                  ]
                }
              ],
              "Footprint": {
                "DesiredAspectRatio": {
                  "x": 1,
                  "y": 1
                }
              },
              "Rooms": [
                {
                  "DesiredShape": {
                    "Area": 25,
                    "AspectRatio": {
                      "x": 1,
                      "y": 1.2
                    }
                  },
                  "Name": "Bedroom",
                  "Type": "Bedroom"
                },
                {
                  "DesiredShape": {
                    "Area": 30,
                    "AspectRatio": {
                      "x": 1,
                      "y": 1.5
                    }
                  },
                  "Name": "Livingroom",
                  "Type": "Living"
                },
                {
                  "DesiredShape": {
                    "Area": 10,
                    "AspectRatio": {
                      "x": 1,
                      "y": 1.5
                    }
                  },
                  "Name": "Bathroom",
                  "Type": "Bathroom"
                },
                {
                  "DesiredShape": {
                    "Area": 15,
                    "AspectRatio": {
                      "x": 1.5,
                      "y": 1
                    }
                  },
                  "Name": "Kitchen",
                  "Type": "Kitchen"
                }
              ]
            },
            "Interior": {
              "Flooring": {
                "MaterialSets": [
                  {
                    "Name": "Floorboard roomtypes",
                    "SampleSet": {
                      "MaterialTypes": [
                        "Floorboards"
                      ]
                    },
                    "TargetSet": {
                      "RoomTypes": [
                        "Kitchen"
                      ]
                    }
                  },
                  {
                    "Name": "Carpet roomtypes",
                    "SampleSet": {
                      "MaterialTypes": [
                        "Carpet"
                      ]
                    },
                    "TargetSet": {
                      "RoomTypes": [
                        "Living",
                        "Bedroom"
                      ]
                    }
                  },
                  {
                    "Name": "Bathroom",
                    "SampleSet": {
                      "MaterialTypes": [
                        "Parquetry"
                      ]
                    },
                    "TargetSet": {
                      "RoomNames": [
                        "Bathroom"
                      ]
                    }
                  }
                ]
              },
              "Furniture": {
                "FurnitureArrangements": [
                  {
                    "DesiredSpatialDensity": "Dense",
                    "Name": "Dense furniture roomtypes",
                    "TargetSet": {
                      "RoomTypes": [
                        "Living",
                        "Bedroom",
                        "Kitchen",
                        "Bathroom"
                      ]
                    }
                  }
                ]
              },
              "Walls": {
                "MaterialSets": [
                  {
                    "Name": "Brick roomtypes",
                    "SampleSet": {
                      "MaterialTypes": [
                        "Brick"
                      ]
                    },
                    "TargetSet": {
                      "RoomTypes": [
                        "Living"
                      ]
                    }
                  },
                  {
                    "Name": "Tiles roomtypes",
                    "SampleSet": {
                      "MaterialTypes": [
                        "Tiles"
                      ]
                    },
                    "TargetSet": {
                      "RoomTypes": [
                        "Bathroom"
                      ]
                    }
                  }
                ]
              }
            }
          }
        ]
      }
    ]
}

Run the following command in the AWS CLI to create a template named bedroom-house defined by the template JSON file you created preceding. You can also perform the same action via AWS RoboMaker console user interface following the blog here:

aws robomaker create-world-template --name single-bedroom-house --template-body file://template.json

Export the ARN from the response to an environment variable called TEMPLATE_ARN. You will use it to create a batch of simulations later in the blog. The response looks like the following.

{
    "arn": <TEMPLATE_ARN>,
    "clientRequestToken": <TOKEN>,
    "createdAt": "2020-09-28T00:14:45+00:00",
    "name": "single-bedroom-house",
    "tags": {}
}

Run the following command to export the ARN to an environment variable, and be sure to replace the placeholder TEMPLATE_ARN with the actual ARN from the response

export TEMPLATE_ARN=<TEMPLATE_ARN>

The preceding JSON world template body document passed in the API creates a template that makes the generated worlds look like a variation of the following.

AWS WorldForge simulation world

Next, generate WorldForge worlds from this template from the AWS CLI. You can select to have different combinations of furniture with a same floor plan, emulating different furniture and their placements, or have different floor plans itself. In this blog, we have five worlds each, for two floor plan variation, taking the total worlds to ten. The default limit for number of WorldForge worlds for each generation job is 50. The total worlds can be in combination of one floor plan with 50 worlds (1×50) or 50 floor plans with one world each (50×1) or any combination in between.

Run the following API command from the AWS CLI to create the worlds.

aws robomaker create-world-generation-job --template $TEMPLATE_ARN --world-count floorplanCount=2,interiorCountPerFloorplan=5

Export the ARN from the output of the preceding API call to an environment variable called GENERATION_JOB_ARN. You are using it to create a batch of simulations later within this blog. The response will represent the following.

{
    "arn": <GENERATION_JOB_ARN>,
    "status": "Pending",
    "createdAt": "2020-10-20T15:53:46+00:00",
    "clientRequestToken": <TOKEN>,
    "template": <TEMPLATE_ARN>,
    "worldCount": {
        "floorplanCount": 2,
        "interiorCountPerFloorplan": 5
    },
    "tags": {}
}

Export the ARN to an environment variable called GENERATION_JOB_ARN using the following command.

export GENERATION_JOB_ARN=<GENERATION_JOB_ARN>

Find out the status of the generation job by running the following command.

aws robomaker describe-world-generation-job --job $GENERATION_JOB_ARN

The world generation command takes minutes to complete.

You can run the describe command occasionally to check the progress. Once the status of the generation job is “Completed”, the response represents the following.

{
    "arn": <GENERATION_JOB_ARN>,
    "status": "Completed",
    "createdAt": "2020-09-28T00:21:02+00:00",
    "clientRequestToken": <TOKEN>,
    "template": <TEMPLATE_ARN>,
    "worldCount": {
        "floorplanCount": 2,
        "interiorCountPerFloorplan": 5
        },
    "finishedWorldsSummary": {
        "finishedCount": 10,
        "succeededWorlds": [
            <WORLD_1_ARN>,
            <WORLD_2_ARN>,
            <WORLD_3_ARN>,
            <WORLD_4_ARN>,
            <WORLD_4_ARN>,
            <WORLD_5_ARN>,
            <WORLD_6_ARN>,
            <WORLD_7_ARN>,
            <WORLD_8_ARN>,
            <WORLD_9_ARN>,
            <WORLD_10_ARN>,
            ]
        },
    "tags": {}
 }

You will run the describe-world-generation command once again, and writing the output to a file later within the blog, when creating the configuration for the batch API.

You can view the generation job using the console at the following URL.

https://console.aws.amazon.com/robomaker/home#worldGenerationJobs

The preceding generation job completed in a minute for the configuration provided and resulted in the worlds that look like the following. You can see five variations of furniture, flooring and other parameters in each of the two floor plans.

 

AWS RoboMaker simulation environments with variations of furniture, flooring, and other parameters.

To recap, by now you have:

  • Created a simulation world template in WorldForge, which contains the parameters and components used to determine the world to be generated.
  • Started a generate world job, specifying the number of variations of floor plans, and number of worlds within each floor plan
  • Made an API call to describe the generation job, which upon completed gives the WORLD_ARN’s of all the generated worlds
  • Exported all the respective ARNs to environment variables.

All the ten 3D WorldForge words were generated with 3 API calls and completed within a couple of minutes! Saving time and effort compared to creating worlds manually in an editor. Let’s now look at the next section, where we learn to prepare the sample application.

Preparing the sample application for AWS RoboMaker:

Clone and build the application using the following commands:

git clone https://github.com/aws-samples/aws-robomaker-worldforge-mapping-evaluation-sample-app.git
cd aws-robomaker-worldforge-mapping-evaluation-sample-app/simulation_ws
git checkout version_1
rosws update
rosdep update
rosdep install -y --from-path src --ignore-src
colcon build
colcon bundle

The preceding block creates the bundle required for running the application in AWS RoboMaker service at simulation_ws/bundle/output.tar location.

Test the application on your local machine by running the following commands:

source install/setup.bash
export TURTLEBOT3_MODEL="waffle_pi"
export BUCKET_NAME="<YOUR_BUCKET_NAME>"
roslaunch simulation_app mapping_sample_application.launch gui:=true x_pos:=3.5 y_pos:=2.0

You see a Turtlebot moving in the gazebo environment building a map! The application is set up to run on a default gazebo environment, and can be switched to WorldForge mode by setting the environment variable GAZEBO_WORLD to WORLDFORGE. Setting the environment variable to WORLDFORGE on the application requires you to export the WORLDFORGE world to your local machine. You can find instructions for exporting the world here.

AWS RoboMaker has first class support for WorldForge worlds so you can connect the application to the WorldForge world directly on AWS RoboMaker which is covered further within this blog. Exit the local simulation here.

Create the Simulation Application in AWS RoboMaker, by uploading the bundle file from your local machine to an Amazon S3 bucket. Export the S3 bucket name in your AWS account to the environment variable BUCKET_NAME on your local machine. Use “sim_app_path” prefix on your Amazon S3 bucket to upload the bundle file.

export BUCKET_NAME=<YOUR_BUCKET_NAME>
export SIM_APP_PREFIX="sim_app_path"

Run the following command to copy the bundle file to an Amazon S3 bucket, and then to create the Simulation Application.

aws s3 cp bundle/output.tar s3://$BUCKET_NAME/$SIM_APP_PREFIX/output.tar
aws robomaker create-simulation-application --name my-mapping-app --sources s3Bucket=$BUCKET_NAME,s3Key=$SIM_APP_PREFIX/output.tar,architecture=X86_64 --robot-software-suite name=ROS,version=Melodic --simulation-software-suite name=Gazebo,version=9 --rendering-engine name=OGRE,version=1.x

Export the ARN from the response to an environment variable called SIM_APP_ARN. This is also used when creating the batch of simulation jobs.

export SIM_APP_ARN=<SIM_APP_ARN>

Running a Batch of Simulations with WorldForge Worlds

By now, you should have a simulation application setup in AWS RobMaker. Create a batch of ten simulation jobs, to test the mapping algorithm in ten different WorldForge worlds that were generated. More variety in worlds provides robust testing for algorithms in simulation environments.

Create the configuration file for the batch API using create_batch_params.py script in sample application repository. You must export additional environment variables to get the network settings for your simulation right.

Find out the default Virtual Private Cloud (VPC) Identity (ID) in your AWS account by running the following command.

sudo apt install jq -y
aws ec2 describe-vpcs | jq '.Vpcs[] | select(.IsDefault == true) | .VpcId'

Export the output which gives the ID of the default VPC, to an environment variable referred by  DEFAULT_VPC_ID by running the following command

export DEFAULT_VPC_ID=<DEFAULT_VPC_ID>

Find out the corresponding subnet’s associated with the default VPC by running the following command.

aws ec2 describe-subnets | jq '.Subnets[] | select(.VpcId == env.DEFAULT_VPC_ID) | .SubnetId'

Pick any two subnets from the output and export them to local environment variables using the following command:

export SUBNET_1=<SUBNET_1>
export SUBNET_2=<SUBNET_2>

Locate the ID of the default Security Group in your account by running the following command.

aws ec2 describe-security-groups | jq '.SecurityGroups[] | select (.GroupName=="default") | select (.VpcId==env.DEFAULT_VPC_ID) | .GroupId'

Export the output to an environment variable referred by SECURITY_GROUP_ID

export SECURITY_GROUP=<SECURITY_GROUP_ID>

Next, create an IAM role to run your AWS RobMaker simulation jobs.

Save the following JSON to a file called trust_policy.json, which will be the role policy associated with the IAM role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "robomaker.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Now you have everything needed to create the IAM role. Run the following commands to create the role.

export IAM_ROLE_NAME=mapping-role
aws iam create-role --role-name $IAM_ROLE_NAME --assume-role-policy-document file://trust_policy.json

Export the ARN from the output to the environment variable IAM_ROLE_ARN with the following command.

export IAM_ROLE_ARN=<IAM_ROLE_ARN>

Attach the managed AWSRoboMaker_Access to the role with the following command. This gives the IAM role permission to act on any AWS RobMaker API.

aws iam attach-role-policy --role $IAM_ROLE_NAME --policy-arn arn:aws:iam::aws:policy/AWSRoboMaker_FullAccess

Save the following JSON file to a file called mapping_robomaker_policy.json. Ensure that you have replaced <BUCKET_NAME> with the bucket in the environment variable BUCKET_NAME. This file is used to provide the custom permission the IAM role needs to write logs , access the application bundle and to upload maps to Amazon S3 bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "cloudwatch:PutMetricData",
                "logs:DescribeLogStreams",
                "logs:CreateLogGroup",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::<BUCKET_NAME>/*"
        }
    ]
}

Create a custom policy and store the ARNs to environment variables by running the following commands

aws iam create-policy --policy-name mapping-robomaker-policy --policy-document file://mapping_robomaker_policy.json

Export the ARN to an environment variable referred by S3_POLICY_ARN

export MAPPING_POLICY=<MAPPING_POLICY_ARN>

Attach the custom policy with the IAM role with the following command

aws iam attach-role-policy --role $IAM_ROLE_NAME --policy-arn $MAPPING_POLICY

You have everything needed to create the configuration for the batch API. Create the configuration file and start a batch of simulations by running the following commands.

cd ../../aws-robomaker-worldforge-mapping-evaluation-sample-app/setup_scripts
aws robomaker describe-world-generation-job --job $GENERATION_JOB_ARN --output json > generation_job_output.json
#Read world ARN's from  'generation_job_output.json';write config to 'batch_config.json'
python create_batch_params.py
aws robomaker start-simulation-job-batch --cli-input-json file://batch_config.json

The simulations are created from this API. Open the Simulation Application tool from the simulation details page by navigating to a simulation job on the AWS RoboMaker console and clicking Connect to set the robot navigating and mapping the world.

AWS RAWS RoboMaker simulation job

Once the robot explores the area and creates the map, the map files are saved from the simulations are uploaded to the Amazon S3 location s3://$BUCKET_NAME/map_results by default. Use the following ROS node that sets the termination criteria for the mapping run, writes the map to disk, and sends a request to upload it to Amazon S3. Tune the parameters for the node in the launch file here. Relevant section from the launch file is following:

<node pkg="simulation_app" type="file_uploader.py" name="file_uploader" >
       <param name="ROBOT_STOP_TIMEOUT"  type="double" value="60"/>
       <param name="TOTAL_MAPPING_TIMEOUT"  type="double" value="900"/>
       <param name="NORM_ONE_DISTANCE_THRESHOLD"  type="double" value="0.1"/>
       <param name="S3_PREFIX_PATH"  type="str" value="map_results/"/>
       <param name="LOCAL_WRITE_PATH"  type="str" value="/tmp/"/>
       <!-- Do not modify the simulation_job_id env variable. Its pulled automatically from RoboMaker Simulations -->
       <param name="AWS_ROBOMAKER_SIMULATION_JOB_ID"  type="str" value="$(optenv AWS_ROBOMAKER_SIMULATION_JOB_ID default_job_id)"/>
  </node>

You have now completed all tasks. You have all the maps from the simulation runs available in a single location, named after the simulation-ids to refer to the logs and data from the AWS RobMaker console.

Shown below is an image with a generated map downloaded from the Amazon S3 bucket.

Map of testing area

The image below shows all the map files available in the Amazon S3 bucket.

Map files in S3 bucket

Summary:

In this blog, you dove into how to create a variety of world’s in AWS RobMaker WorldForge in minutes. You also identified how to run simulations in parallel by connecting a sample application to the generated worlds, and running them concurrently. Robotics developers can now test core algorithms, by setting up their applications to work with the WorldForge worlds within a couple of quick API calls!

To learn more about AWS RoboMaker, visit https://aws.amazon.com/robomaker/ or contact us at aws-robomaker-bd@amazon.com