Build an Alexa controlled robot with AWS RoboMaker
Alexa is Amazon’s cloud-based voice service and Alexa-compatible devices allow users to control their smart devices with voice commands. Today, Alexa is available on hundreds of millions of devices from Amazon echo devices to third-party devices. Alexa Skill Kit (ASK) provides a way for robot developers and manufacturers to build a natural voice interface for their robots. AWS RoboMaker is a cloud robotics solution that provides a customized development environment, a fully-managed simulation infrastructure, and built-in cloud extensions making it easy to build and test robot applications and integrate them with other AWS services. In this blog post, I will show you how to use AWS RoboMaker to develop and test an Alexa-controlled robot.
Overview of Solution
Alexa provides a set of built-in capabilities, referred to as skills. A robot controller skill receives the customer’s voice command through an Alexa enabled device like Amazon Echo and converts the command to a text message. The text message is published to an IoT topic in an Alexa skill AWS Lambda code hook. The robot receives the text command through the subscribed AWS IoT topic and executes the command. AWS RoboMaker is used to develop and simulate the voice-controlled robot.
Building an Alexa-controlled robot in AWS RoboMaker involves the following steps:
- Create an IoT thing in AWS IoT.
- Create an Alexa skill.
- Create a robot application in AWS RoboMaker.
- Simulate the robot in AWS RoboMaker simulations.
For this walk through, you need to have the following prerequisites:
- An AWS account
- An Alexa developer account
- A Region that supports AWS RoboMaker. This blog uses us-west-2 (Oregon) Region.
Step 1: Create a IoT thing in AWS IoT
First, create an IoT thing in AWS IoT Core Service. The IoT thing represents a robot in AWS IoT, and the command sends from Alexa to the robot through the IoT topic. For more details, see Create a thing.
- Log in to AWS IoT Core Console, select Onboard, then select Get Started. Click Onboard a device.
- Select Linux/OSX as your platform and select Python as the AWS IoT Device SDK, then click Next.
- Create a name for your IoT thing. For example: VoiceControlledRobot, then click Next.
- Select Linux/OSX to download the IoT connection kit package. Use the IoT thing certificates in the downloaded package to create the IoT MQTT client in the following steps. Click Next Step.
- Review the Configure and test your device section, then click Done.
- Go to AWS IoT Core, select Manage and then select Things. Select Create IoT thing and select Interact. Save the Rest API Endpoint of the IoT thing. The IoT thing endpoint is used to create the IoT MQTT client in the following steps.
Step 2: Create an Alexa skill
Here we build a new robot skill that converts voice commands to text message and sends the command to the robot. For more details, see Build Skills with the Alexa Skills Kit.
- Log in to Alexa Developer Console, then click Create Skill.
- Create a skill name, and select Alexa-Hosted(Python) as the method to host your skill’s back end resources, then click Create Skill.
- Select default Hello World Skill as template, then select Continue with template to create a new skill.
- Select + Add Intent in the intents section to create a name for the new intent, for example, MoveForwardIntent.
- Select Create Custom Intent and create a new MoveForwardIntent.
Add sample utterances to MoveForwardIntent:
tell robot to move forward
- Select Save Model and then select Build Model.
- Select the Code tab at the top of the page. Add Amazon Root CA1 and AWS IoT Certificates downloaded in Step 1 to lambda folder. These certificates are used to build the IoT MQTT connection’
8. Add IoT MQTT Client and intent handler to existing lamba_funtion.py.
This sample Python script shows how to publish the command to an IoT Topic in MoveForwardIntent Handler. For more details about MQTT, see here.
from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTClient # Parameters for AWS IoT MQTT Client. iotThingEndpoint = "<iot_host_endpoint>" # IoT thing endpoint saved in Step 1. iotThingPort = 8883 rootCAPath = "<root-CA_path>" # eg. "./certificates/AmazonRootCA1.pem" privateKeyPath = "<private_key_path>" # eg. "./certificates/VoiceControlledRobot.private.key" certificatePath = "<certificate_key_path>" # eg. "./certificates/VoiceControlledRobot.cert.pem" iotTopic = "topic_1" # Init AWSIoTMQTTClient skillIoTMQTTClient = AWSIoTMQTTClient("sdk-nodejs-publisher") skillIoTMQTTClient.configureEndpoint(iotThingEndpoint,iotThingPort) skillIoTMQTTClient.configureCredentials(rootCAPath, privateKeyPath, certificatePath) # AWSIoTMQTTClient connection configuration skillIoTMQTTClient.configureAutoReconnectBackoffTime(1, 32, 20) skillIoTMQTTClient.configureOfflinePublishQueueing(-1) # Infinite offline Publish queueing skillIoTMQTTClient.configureDrainingFrequency(2) # Draining: 2 Hz skillIoTMQTTClient.configureConnectDisconnectTimeout(10) # 10 sec skillIoTMQTTClient.configureMQTTOperationTimeout(5) # 5 sec # Connect and subscribe to AWS IoT skillIoTMQTTClient.connect() logger.info("mqtt connected") class MoveForwardIntentHandler(AbstractRequestHandler): """Handler for Move Forward Intent.""" def can_handle(self, handler_input): # type: (HandlerInput) -> bool return ask_utils.is_intent_name("MoveForwardIntent")(handler_input) def handle(self, handler_input): # type: (HandlerInput) -> Response speak_output = "Ok, move forward" skillIoTMQTTClient.publish(iotTopic, "forward", 1) return ( handler_input.response_builder .speak(speak_output) .response )
- Repeat previous Step 2 to add a MoveBackwardIntent.
Step 3: Create a robot application with AWS RoboMaker
- Go to AWS RoboMaker console, select Development, and then select Robot applications. Click Create robot application.
- Enter a name for the robot application. For ROS distribution, select ROS Kinetic. Use the default VPC and select a subnet. Select Create to create the AWS Cloud9 development environment.
- In the AWS Cloud9 development environment, select AWS RoboMaker Resources, then select Download samples, and then select Hello World.
- On the left, in the Environment tab, expand HelloWorld, HelloWorld, robot_ws, src, hello_world_robot. Create certificates fold and upload Amazon Root CA1 and AWS IoT Certificates downloaded in Step 1 to this folder. These certificates are used to build the IoT MQTT connection.
- Add AWS IoT dependencies to ROS.
Open a new terminal in AWS RoboMaker Development environment and run following command:
pip install AWSIoTPythonSDK sudo bash -c 'echo "yaml https://s3.us-west-2.amazonaws.com/rosdep/iot.yaml" > /etc/ros/rosdep/sources.list.d/21-aws-iot-pip.list' cd ~/environment/HelloWorld/robot_ws rosdep update cd ~/environment/HelloWorld/simulation_ws rosws update rosdep install --from-paths src --ignore-src -r -y colcon build colcon bundle
- Create a new node to receive MQTT command from Alexa and create a mqtt_listener.launch file for new created node.
This sample MQTT listener node script shows how to subscribe and receive command from IoT Topic.
import rospy import logging import time from geometry_msgs.msg import Twist from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTClient # Configure logging logger = logging.getLogger("AWSIoTPythonSDK.core") logger.setLevel(logging.WARN) # Parameters for AWS IoT MQTT Client. iotThingEndpoint = "<iot_host_endpoint>" iotThingPort = 8883 certificatePath = os.environ["CERTIFICATES"] rootCAPath = certificatePath + "<root-CA_filename>" # eg. "AmazonRootCA1.pem" privateKeyPath = certificatePath + "<private_key_filename>" # eg. "VoiceControlledRobot.private.key" certificatePath = certificatePath + "<certificate_key_filename>" # eg. "VoiceControlledRobot.cert.pem" iotTopic = "topic_1" # Init AWSIoTMQTTClient skillIoTMQTTClient = AWSIoTMQTTClient("sdk-nodejs-subscriber") skillIoTMQTTClient.configureEndpoint(iotThingEndpoint,iotThingPort) skillIoTMQTTClient.configureCredentials(rootCAPath, privateKeyPath, certificatePath) # AWSIoTMQTTClient connection configuration skillIoTMQTTClient.configureAutoReconnectBackoffTime(1, 32, 20) skillIoTMQTTClient.configureOfflinePublishQueueing(-1) # Infinite offline Publish queueing skillIoTMQTTClient.configureDrainingFrequency(2) # Draining: 2 Hz skillIoTMQTTClient.configureConnectDisconnectTimeout(10) # 10 sec skillIoTMQTTClient.configureMQTTOperationTimeout(5) # 5 sec # Connect and subscribe to AWS IoT skillIoTMQTTClient.connect() DONE = False # Initialize the node rospy.init_node('listener', anonymous=True) cmd_pub = rospy.Publisher('/cmd_vel', Twist, queue_size=10) # Custom MQTT message callback def customCallback(client, userdata, message): global DONE rospy.loginfo("Received payload!") rospy.loginfo(message.topic) rospy.loginfo(message.payload) command = message.payload rospy.loginfo("Processing command: " + command) # Execute command if command == "forward": twist = Twist() twist.linear.x = 1 cmd_pub.publish(twist) time.sleep(1) twist.linear.x = 0 cmd_pub.publish(twist) elif command == "backward": twist = Twist() twist.linear.x = -1 cmd_pub.publish(twist) time.sleep(1) twist.linear.x = 0 cmd_pub.publish(twist) else: DONE = True # Subscribe to topics skillIoTMQTTClient.subscribe(iotTopic, 1, customCallback) time.sleep(2) while True: time.sleep(1) if DONE: exit(0) skillIoTMQTTClient.unsubscribe(iotTopic)
Here is the sample launch file.
<launch> <arg name="use_sim_time" default="true"/> <param name="use_sim_time" value="$(arg use_sim_time)"/> <!-- Start MQTT listener on launch --> <node name="listener" pkg="hello_world_robot" type="listener.py" output="screen"> <env name="CERTIFICATES" value="$(find hello_world_robot)/certificates/" /> </node> </launch>
- Build and bundle the robotics application sources
cd ~/environment/HelloWorld colcon build colcon bundle Open a new terminal, copy the robot application source bundle to your Amazon S3 bucket. aws s3 mb s3://<your-bucket-name> aws s3 cp ~/environment/HelloWorld/robot_ws/bundle/output.tar s3://<your-bucket-name>/robot.tar aws s3 cp ~/environment/HelloWorld/simulation_ws/bundle/output.tar s3://<your-bucket-name>/simulation.tar
- Create a robot application.
aws robomaker create-robot-application --name MyRobotApplication --sources s3Bucket=<your-bucket-name>,s3Key=robot.tar,architecture=X86_64 --robot-software-suite name=ROS,version=Kinetic
This command returns information about the newly created robot application.
- Create a simulation application.
aws robomaker create-simulation-application --name MySimulationApplication --sources s3Bucket=<your-bucket-name>,s3Key=simulation.tar,architecture=X86_64 --robot-software-suite name=ROS,version=Kinetic --simulation-software-suite name=Gazebo,version=7 --rendering-engine name=OGRE,version=1.x
The command returns information about the newly created simulation application.
Step 4: Simulate the Robot with AWS RoboMaker Simulations
- Create a simulation job role by following our AWS document here.
- Go to AWS RoboMaker console, select Simulation Jobs, and click Create simulation job.
- For ROS distribution, select ROS Kinetic. For IAM role, select the role created in step 1.
- For Networking, select default VPC, default security groups, and two subnets. Select Next.
- For Robot application, select MyRobotApplication. Usehello_world_robot as launch package name and use mqtt_listener.launch as launch file name. Click Next.
- For Simulation application, use hello_world_simulation as launch package name and use empty_world.launch as launch file. Click Next.
- Review the details and click Create to create a simulate job.
- Select the new create simulation job and connect to Gazebo. A simulation window opens up.
- Go to Alexa Developer Console, select the new created skill, and select Test.
- Enter “Ask <invocation name> to move forward” and the robot moves forward in the simulation window.
In this blog, we demonstrated and walked you through the process on how to build a voice-controlled robot with an Alexa skill kit and AWS RoboMaker. To learn more about AWS IoT, visit https://aws.amazon.com/iot/. To learn more about Alexa Skill Kit, visit https://developer.amazon.com/en-US/alexa/alexa-skills-kit. To learn more about AWS RoboMaker, visit https://aws.amazon.com/robomaker/.