iRobot Uses AWS RoboMaker to Speed Testing, Deliver More-Effective Robots

iRobot chose to move its simulation testing environment to the Amazon Web Services (AWS) Cloud to help solve its challenges. “We were already an AWS customer, using AWS IoT services to monitor our robot fleet,” Kruger says. “We trust AWS to deliver reliability, flexibility, and scalability.”

The company began using AWS RoboMaker, a service that helps developers build, test, and deploy robotics applications in the cloud. iRobot relies on the service to discover problems across different product lines via automated regression testing. Using the AWS RoboMaker simulation service, iRobot can test each pull request against different scenarios and robot types.

iRobot also takes advantage of AWS Lambda and AWS Step Functions to bridge its continuous integration/continuous delivery (CI/CD) pipeline with AWS RoboMaker to run multiple simulations in parallel for various home layouts and floor types. “When a developer submits new code, it goes through a broad spectrum of simulations on AWS RoboMaker,” says Kruger. “That ranges from basic tests like cleaning a room to more interactive tests like putting a robot in a room, having it run a mission, and then figuring out how to get back to its base.”

Using AWS RoboMaker, iRobot can test across all five of its newer floor care products simultaneously and perform tests faster than it could using its on-premises solution. “We are able to complete tests faster using AWS RoboMaker compared to our physical testing solution,” Kruger says. “We can validate scenarios in hours instead of weeks, and we can find code quality issues in simulation testing that we were unable to catch earlier.” Previously, iRobot would run a series of smoke and regression tests before moving code to production. “Smoke tests would take a few hours, and regression tests could take weeks,” says Kruger. “Now, with simulation testing on every code submission, there is a higher quality baseline. We also see higher pass rates on smoke tests and fewer resets of regression testing. All of this speeds our software time-to-market.”

iRobot now performs daily tests in parallel on AWS RoboMaker. “Running 40 tests in parallel is like having 20 extra testers,” says Kruger. “In the past, we could test a build against one robot to validate functionality. Now, we can test it across our entire product line.” Within the first three months of using AWS RoboMaker, iRobot was executing 5,000 test scenarios a month, where previously the company might do 100 tests monthly.

It can also perform more in-depth simulation tests once changes and pull requests are merged. “We used to run a test in an environment two or three times to test product reliability. Using the AWS RoboMaker simulator, we can run tests faster than real time and in parallel, enabling us to run 20 times the number of tests we did before, which gives us a more comprehensive measure of the product’s stability,” says Kruger. “We’re also moving from broad spectrum testing to more targeted testing.”

By taking advantage of its new testing solution built on AWS, iRobot can detect software defects in development instead of production. “Our developers can more easily determine whether change requests were successful or if they caused errors,” says Kruger. As a result, the company is producing higher-quality code and ultimately delivering better, more-effective products to customers. “Running tests in parallel gives developers faster feedback so they can find problems quicker. That means we can get new software updates out to customers in less time. This is going to lead to better, more-effective robots.”

iRobot continues to use AWS RoboMaker to add more test scenarios. “We see AWS RoboMaker as something we can build on,” says Kruger. “It helps us move faster and it will enable us to keep focusing on adding more functionality to our products, making them even smarter.”

To learn more, visit aws.amazon.com/robomaker.