AWS Mobile Blog

AWS CodePipeline adds support for AWS Device Farm as a test provider

AWS CodePipeline now supports AWS Device Farm as a test provider to deliver truly automated testing in your continuous integration and continuous delivery (CI/CD) pipeline.

In a previous blog post “Build a CI/CD Pipeline for Your Android App with AWS Services,” Vinay Selvaraj walked you through building a complete CI/CD pipeline using AWS CodeCommit, AWS CodeBuild, AWS Lambda, AWS Device Farm, and Amazon S3 (all triggered by a single code repository push).

In that solution, two Lambda functions were deployed-one to provision a Device Farm test project, and the other to kick off a Device Farm test on real devices. Now that Device Farm is fully integrated as a test provider for AWS CodePipeline, we can greatly improve on that solution.

In this article, we first walk you through adding Device Farm as a test provider directly into your build, test, and deployment pipeline using the CodePipeline console. We then run through building an entire CI/CD pipeline solution by using a single CloudFormation template. The template automates resource provisioning with Device Farm as a native test provider in the test stage of a CodePipeline pipeline.

Let’s go over the new addition to CodePipeline.

Add Device Farm as a test provider using the AWS CodePipeline console

Start by signing into the CodePipeline console.

If you don’t already have an existing CodePipeline pipeline created, follow these instructions to create one. Then follow the steps below to add Device Farm as a test provider.

1.     Select an existing pipeline from the CodePipeline Console.

2.     Choose Edit at the top of your existing pipeline

3.     Add a new stage in your pipeline and name your stage

4.     Choose Action. This opens a new window on the right half of the console.

5.     In the new action window, for Action Category, choose Test.

6.     Enter an Action name: (for example, TestOnRealDevices).

7.     For Test Provider, choose AWS Device Farm.

8.     For Project name, choose if you want to use an existing Device Farm test project or create a new one

Existing Device Farm test project

If you have an existing Device Farm test project, choose the project name from the dropdown and skip to step 10.

New Device Farm test project

You can create a new Device Farm project by choosing Create a new project. This directs you to the Device Farm console to provision a new project. After you create a new project via the Device Farm console, return here, hit the refresh button (located to the right of the Project name field), and then choose the newly created project name from the dropdown.

9.     Configure the remaining required and optional properties.

10.  Choose Add action to save this test stage.

Congratulations! You’ve successfully added Device Farm as a test provider to your CodePipeline stage using the console. You can now trigger the pipeline with a code push, and CodePipeline will integrate directly with Device Farm to test the complied app on real devices.

Now, let’s run through the end-to-end automated solution using AWS CloudFormation.

Build an end-to-end CI/CD pipeline solution using AWS CloudFormation

This solution includes a pre configured AWS CloudFormation template. You use the template to build an AWS CloudFormation stack. The stack automates the provisioning of all the resources you need for an end-to-end build, test, and deployment CI/CD pipeline for a mobile Android app.

You use the AWS CloudFormation template to provision the following resources on your account:

  • A new CodeCommit repo to be used as a trigger to build your app using CodeBuild after a git push command.
  • A new CodeBuild project that’s triggered by CodeCommit to compile, package, and deliver your Android app.
  • A new AWS Lambda function that creates and configures a new Device Farm project and device pool to test your Android app on real physical devices.
  • A new S3 bucket to store build artifacts.
  • A new CodePipeline pipeline to facilitate and monitor each build, test, and deployment phase for your Android app from each commit to app delivery.

Note: In addition to provisioning these resources, the template integrates the services inside your CodePipeline pipeline to trigger the stages and write to the correct S3 bucket location.

  1. Download the AWS CloudFormation template here.
  2. Launch the AWS CloudFormation console.
  3. Choose Create Stack button.
  4. Choose Choose File under Choose a template > Upload a template to Amazon S3, then choose the template you downloaded previously, then choose Next.
  5. Provide a stack name (for example, myapp-cicd-pipeline).
  6. Review the default parameters, and then click Next. (Note that if your Android app module isn’t named ‘app’, you need to change the AppModuleName parameter.)
  7. (Optional) Set any options that you need in Options, and then click Next.
  8. Select I acknowledge that AWS CloudFormation might create IAM resources.  (The stack uses IAM roles to manage Lambda, Device Farm, CodeBuild, and CodePipeline.)
  9. Choose Create button.
  10. Once the stack shows the ‘CREATE_COMPLETE’ status, launch the CodePipeline console.
  11. Select pipeline and choose Edit.
  12. Under the Test stage, choose Edit (the pencil)
  13. Provide the Project name, Device pool, and App file path as shown. Project name and Device pool will auto populate for you
  14. Choose Update.
  15. Choose Save pipeline changes (blue button at the top).

Now that your resources are all provisioned, we need to configure an Android project to use the newly created CodeCommit repo. Remember, after this is all set up, the entire CI/CD pipeline gets kicked off automatically each time you commit Android project code changes to the CodeCommit repo. This saves your team a ton of tedious steps each time you want to test and release a new feature.

Setup Git and configure your Android project to use CodeCommit

Note: If you want to quickly test a pipeline without setting up your own Android project, you can use our Android Notes Tutorial app to get started. We used that app project code to test the integration of the pipeline.


If your development environment isn’t already configured for CodeCommit, follow the CodeCommit setup steps that walk you through installing the Git client, granting IAM permissions to push code, and setup the credentials helper. You can skip step 4 because we already created a CodeCommit repo via the CloudFormation stack.

Configure your Android project with CodeCommit

To configure your Android project to use CodeCommit, we’ll need to add the CodeCommit repository that was created by the AWS CloudFormation stack previously. To do this, copy the CodeRepoCloneUrlHttp from the AWS CloudFormation stack output value (example Outputs tab below), and run the below git command from within your Android project workspace:

Figure 1

git remote add codecommit <CodeRepoCloneUrlHttp from the stack output>

Pushing code to CodeCommit

This is where the fun begins!  It’s time to push the Android source code to your CodeCommit repository. After you push the code, our cloud automation CI/CD pipeline takes over. After the initial push to the remote repo, you’ll soon have a fully built app that has been tested on real devices and a deployed APK file that’s staged in an S3 bucket.

Make a change to your Android project that’s worthy of building and testing, add those files, commit, and then push to the CodeCommit repo as follows:

Add changed files to your local Git repo:

git add *

Commit the changes to your local Git repository:

git commit -m "Initial push of the code"

Push the changes to your CodeCommit repository:

git push

Watch the training wheels come off

To view the progress of the CodePipeline execution, open the CodePipeline console. Select the pipeline that you previously created, and you’ll see the progress of each pipeline stage.

The execution goes through four stages (Source, Build, Test, and Deliver):

Source: The latest code push is taken from the CodeCommit repository and packaged as a ZIP file for use by the other stages in the pipeline.
Build: CodePipeline triggers a CodeBuild build to compile and package the app as an APK file.
Test: CodePipeline uploads the newly built APK file to the Device Farm project and triggers a test run of your Android app on real devices.
Deliver: The built and tested .apk file is then uploaded to the S3 bucket, and is ready for beta distribution or uploading to the app store or stores.

We hope this solution will save you some cycles in your build, test, and deployment lifecycle. If you want to use GitHub instead of CodeCommit, check out Adrian Hall’s article for integrating GitHub into your CI/CD pipeline.


AWS Device Farm Developer Guide

AWS CodePipeline User Guide