iRobot-Logo@2x

iRobot Uses AWS RoboMaker to Speed Testing, Deliver More-Effective Robots

2020

Millions of people throughout the world use robots from iRobot, such as the popular Roomba, to automate household tasks like vacuuming, mopping, or mowing the lawn. Since its founding in 1990, iRobot has become the world’s leading consumer robot company, with more than 25 million robots sold and a current connected fleet of five million.

As the company has grown and its product line has expanded, software test scenarios have increased, and the company’s on-premises testing environment has struggled to support the growth. “We spend several months testing hardware and software for each release, and we wanted to decrease software testing time so we could get updates to customers faster,” says Chris Kruger, director of software engineering at iRobot.

Over time, iRobot devices have become more sophisticated. “The Roomba now has a camera that maps out rooms to clean entire floors of homes,” says Kruger. “We’ve set up hundreds of scenarios in our home-testing labs to make sure the robots can handle varying home environments and layouts. However, our testing environment couldn’t scale to keep up with the expanding portfolio of products, and it wasn’t stable enough for us to actually trust the results. We were concerned about code quality.”

Figure 1
CustomerReferences_QuoteMark

Using the AWS RoboMaker simulator, we can run tests faster than real time and in parallel, enabling us to run 20 times the number of tests we did before, which gives us a more comprehensive measure of the product’s stability."

Chris Kruger
Director of Software Engineering, iRobot

Using AWS RoboMaker to Run Multiple Simulations in Parallel

iRobot chose to move its simulation testing environment to the Amazon Web Services (AWS) Cloud to help solve its challenges. “We were already an AWS customer, using AWS IoT services to monitor our robot fleet,” Kruger says. “We trust AWS to deliver reliability, flexibility, and scalability.”

The company began using AWS RoboMaker, a service that helps developers build, test, and deploy robotics applications in the cloud. iRobot relies on the service to discover problems across different product lines via automated regression testing. Using the AWS RoboMaker simulation service, iRobot can test each pull request against different scenarios and robot types.

iRobot also takes advantage of AWS Lambda and AWS Step Functions to bridge its continuous integration/continuous delivery (CI/CD) pipeline with AWS RoboMaker to run multiple simulations in parallel for various home layouts and floor types. “When a developer submits new code, it goes through a broad spectrum of simulations on AWS RoboMaker,” says Kruger. “That ranges from basic tests like cleaning a room to more interactive tests like putting a robot in a room, having it run a mission, and then figuring out how to get back to its base.”

Validating Test Scenarios in Hours Instead of Weeks

Using AWS RoboMaker, iRobot can test across all five of its newer floor care products simultaneously and perform tests faster than it could using its on-premises solution. “We are able to complete tests faster using AWS RoboMaker compared to our physical testing solution,” Kruger says. “We can validate scenarios in hours instead of weeks, and we can find code quality issues in simulation testing that we were unable to catch earlier.” Previously, iRobot would run a series of smoke and regression tests before moving code to production. “Smoke tests would take a few hours, and regression tests could take weeks,” says Kruger. “Now, with simulation testing on every code submission, there is a higher quality baseline. We also see higher pass rates on smoke tests and fewer resets of regression testing. All of this speeds our software time-to-market.”

Scaling to Run 5,000 Tests Monthly

iRobot now performs daily tests in parallel on AWS RoboMaker. “Running 40 tests in parallel is like having 20 extra testers,” says Kruger. “In the past, we could test a build against one robot to validate functionality. Now, we can test it across our entire product line.” Within the first three months of using AWS RoboMaker, iRobot was executing 5,000 test scenarios a month, where previously the company might do 100 tests monthly.

It can also perform more in-depth simulation tests once changes and pull requests are merged. “We used to run a test in an environment two or three times to test product reliability. Using the AWS RoboMaker simulator, we can run tests faster than real time and in parallel, enabling us to run 20 times the number of tests we did before, which gives us a more comprehensive measure of the product’s stability,” says Kruger. “We’re also moving from broad spectrum testing to more targeted testing.”

Catching Defects in Development Instead of Production

By taking advantage of its new testing solution built on AWS, iRobot can detect software defects in development instead of production. “Our developers can more easily determine whether change requests were successful or if they caused errors,” says Kruger. As a result, the company is producing higher-quality code and ultimately delivering better, more-effective products to customers. “Running tests in parallel gives developers faster feedback so they can find problems quicker. That means we can get new software updates out to customers in less time. This is going to lead to better, more-effective robots.”

iRobot continues to use AWS RoboMaker to add more test scenarios. “We see AWS RoboMaker as something we can build on,” says Kruger. “It helps us move faster and it will enable us to keep focusing on adding more functionality to our products, making them even smarter.”

To learn more, visit aws.amazon.com/robomaker.


About iRobot

Based in Bedford, Massachusetts, iRobot is the world’s leading consumer robot company. The company develops and produces the Roomba vacuuming robot and the Braava family of mopping robots, which are used by millions of people across the globe.

Benefits of AWS

  • Validates robot software builds in hours instead of weeks
  • Increases the number of simulated missions by 20 times
  • Runs 5,000 tests in parallel monthly
  • Produces higher-quality code to deliver better products
  • Enables more comprehensive product testing
  • Catches cross-product errors faster via automated regression testing

AWS Services Used

AWS RoboMaker

Robots are being used more widely in society for purposes that are increasing in sophistication such as complex assembly, picking and packing, last-mile delivery, environmental monitoring, search and rescue, and assisted surgery.

Learn more »

AWS Lambda

AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume.

Learn more »

AWS Step Functions

AWS Step Functions lets you coordinate multiple AWS services into serverless workflows so you can build and update apps quickly. Using Step Functions, you can design and run workflows that stitch together services, such as AWS Lambda, AWS Fargate, and Amazon SageMaker, into feature-rich applications.

Learn more »


Get Started

Companies of all sizes across all industries are transforming their businesses every day using AWS. Contact our experts and start your own AWS Cloud journey today.