Front-End Web & Mobile

Automated Device Testing with AWS Device Farm and Jenkins

A strong indicator of quality of an app is its ability to run on multiple devices of different make, model, and manufacturer consistently and predictably. Incremental development models are at the core of every mobile app development team. Often a seemingly benign feature can be disruptive to the overall user experience and functioning of the app if not thoroughly tested. How do you ensure that every code check-in triggers tests for your mobile app against numerous devices and provides detailed analysis of the results?

Continuous integration (CI) systems have been around for a long time and are now at the core of all DevOps workflows. They integrate build, test, and distribution into one workflow. Many systems to implement CI exist, including cloud solutions like AWS CodePipeline and on-premises solutions like Jenkins. Jenkins is an open source CI solution that is easy to install and portable across all major platforms. It has more than 1000 plugins that integrate with build, test, and distribution components.  This makes Jenkins a popular choice among teams looking for a CI tool for a new project.

AWS Device Farm is an app testing service. You can use it to test and interact with your Android, iOS, and web apps on physical phones and tablets that are hosted by AWS. You can use Device Farm for automated testing with a variety of available testing frameworks. You can also use it for manual testing through a remote session.

Today, AWS Device Farm announced the availability of a Jenkins plugin for AWS Device Farm.  The Jenkins plugin enables you to integrate device testing into your CI workflow.

Prepare your AWS account so that Jenkins can access Device Farm resources:

  1. Sign in to the IAM console.
  2. Create or modify an IAM policy so that your Jenkins instance has access to AWS Device Farm.
  3. Create a user for your Jenkins master (with programmatic access) and assign the user to the IAM policy you just created.
  4. Note the access key ID and secret access key. You need them later.

Add the Jenkins plugin to your Jenkins master:

  1. Sign in to your Jenkins web UI.
  2. Click Manage Jenkins > Manage Plugins on the left side of the screen.
  3. Click the Available tab.
  4. Type aws-device-farm in the search box.
  5. Click Install without restart.
  6. After the installation is complete, click Manage Jenkins > Configure System.
  7. Scroll to the AWS Device Farm header.
  8. Paste the access key ID and secret access key into the appropriate boxes.
  9. Click Save.

You can also use the Jenkins CLI to manage plugins.  Now that the Jenkins plugin for AWS Device Farm is available on your Jenkins master, you can use it in a workflow:

  1. Select the project for which you want to set up the plugin.
  2. Add a post-build step: Run Tests on AWS Device Farm.
  3. Configure any options you want for the run. At minimum, select the UI tests that you want to run and the devices you want to run those tests on.
  4. Save the changes for the project.

AWS Device Farm supports major testing frameworks such as Espresso, XCTest UI, Appium, and Calabash.

After a build and test flow is complete, you can see the results of the tests:

Integrating AWS Device Farm into your automated build pipeline enables you to test your mobile applications on more devices, producing a better quality mobile app.  As always, we want to hear from you.  If you have suggestions or issues, let us know on the AWS Forums.