Front-End Web & Mobile

Testing mobile apps across hundreds of real devices with Appium, Node.js, and AWS Device Farm

In this blog post, we’re going to dive into how to use  AWS Device Farm to run and test a sample mobile app using Node.js and Appium.

Device Farm is an application testing service that lets you test and interact with applications on many real devices at once. You can also reproduce issues on a real device in real time. You can use Device Farm to test any type of application—including native iOS and Android, JavaScript (React Native, Ionic, Cordova), Xamarin, and other applications.

With Device Farm, you can view videos, screenshots, logs, and performance data to pinpoint and fix issues. This helps you improve quality before shipping your app. Device Farm lets you test your application on a shared fleet of over 2,500 real devices, or on your own private device fleet in the cloud.

Device Farm recently added support for Node.js. This means you can run Appium, Detox, and any JavaScript testing framework.

Getting started

To get started, you need three things:

·      An AWS account

·      An Appium Node.js test

·      An application .apk or .ipa file

Note: If you don’t have a test already written or an application to test, we provide them for you in the following section. Continue to the next step of creating the demo test.

If you do have a test already written and an .ipa or .apk file, skip to the section “Packaging the test files for Device Farm”.

First, you create a basic test for our application. To do this, you need to create a test file (we’re calling this file test_native_android.js):

var expect = require('chai').expect;

var wd = require('wd'),
    driver = wd.promiseChainRemote({
        host: 'localhost',
        port: 4723
    });


var assert = require('assert');
describe('AWSDeviceFarmReferenceAppTest', function () {


    before(function () {
        this.timeout(300 * 1000);
        return driver.init();
    });


    after(function () {
        console.log("quitting");
    });


    it('test_app_is_loaded', async function () {
        const element = await driver.elementById("com.amazonaws.devicefarm.android.referenceapp:id/homepage_headline");


        expect(element).to.exist;
    });


    it('test_click_on_home_6_times', async function () {
        this.timeout(300 * 1000);
        var i = 5;
        while (i > 0 ) {
            const element = await driver.elementByAccessibilityId("ReferenceApp");
            expect(element).to.exist;
            element.click();


            await new Promise(res => setTimeout(res, 1000));


            const element2 = await driver.elementByXPath("/hierarchy/android.widget.FrameLayout/android.widget.LinearLayout/android.widget.FrameLayout/android.widget.LinearLayout/android.widget.FrameLayout/android.support.v4.widget.DrawerLayout/android.widget.RelativeLayout/android.support.v7.widget.RecyclerView/android.widget.FrameLayout[1]");
            expect(element2).to.exist;
            element2.click();
            i--;
            await new Promise(res => setTimeout(res, 1000));
        }
    });
});

In this test, you’re doing two things:

1.    You first test to see that the application loads.

2.   Next, you run a loop that checks to see if a button exists, chooses the button, and then goes back. The loop runs 5 times.

Note: The code provided is targeting the example .apk that we’re providing you for the tutorial. If you bring your own .apk, you need to write your own test.

Next, you create a package.json file for the dependencies that you need for this test:

{
  "name": "androidsampletest",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "mocha test_web.js"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "chai": "^4.2.0",
    "mocha": "^5.2.0",
    "wd": "^1.11.0"
  }
}

In the package.json, we’ve included the mocha framework (a JavaScript testing framework) and the chai framework (an assertion library for Node.js and the browser). You need these for your test to run, as well as the wd library (a Node.js client for WebDriver).

Next, you need to install the dependencies that are required for this test to run:

npm install

Packaging the test files for Device Farm

Now you need to package the files in this folder into a .tgz file. To do this, first install the npm bundle package:

npm install -g npm-bundle

Next, run the npm bundle command from within the root of your test:

npm-bundle

Now, zip the .tgz file:

zip -r MyTest.zip *.tgz

Your test is now written, bundled, and ready to go.

Finally, export your .apk file if you have an existing application, or download the example Android .apk file.

Running your test on Device Farm

Now that you have a test written and an .apk to run the test on, navigate to the Device Farm console at https://console.aws.amazon.com/devicefarm.

  1. Choose Create New Project, and give the project a name.
  2. Choose Create a New Run.
  3. Choose the Android / iOS button (to specify a native application).
  4. Upload the .apk file.
  5. Give the Run a name, and choose Next Step.

Next, in the dropdown menu for the type of test to run, you can see the option for Appium Node.js. Choose this option, and upload the zipped folder that we created in the previous steps:

Then, you’re given a default TestSpec YAML file that describes some configuration for your test. In the test: phase, you need to update the command: property to be the command that you use to execute your mocha test.

Replace:

node YOUR_TEST_FILENAME.js

with:

./node_modules/mocha/bin/mocha test_native_android.js

Then, choose Save TestSpec.

Note: In this YAML file, you can also set other configuration for your test environment. For example, you can set the node environment that you want to be running in, post-test command , and ADB device commands (like turning off animations).

After the test has been saved, choose Next step.

In the next screen, you’re given the option to choose a custom set of devices (a device pool) or a default device pool. Either choose the default device pool, or create your own by using the devices that you want to test on.

After you’ve chosen your device pool, choose Next step.

In the next screen, you’re given some options to specify the device state. You can set custom device settings that you’d like to configure on each device (things like geolocation, Wi-Fi status, etc.). Either keep the defaults or configure your custom device state, and then choose Next step.

Now, the test is ready to run. Choose Confirm & start run.

After the test is complete, you should see the test results overview in the dashboard. To view the details of the test, choose the test name.

In this screen, you can view a test overview for each device. You can choose an individual device to view details of the test.

After choosing an individual device, you’re taken to the device details page. You can view details of the individual test, including videos of the test, test logs, app performance metrics, and more.

Next steps

To learn more about Device Farm, check out the getting started information and documentation.

To view all available devices in Device Farm, see the devices list.