Automate Testing with Authentication using AWS Amplify and Cypress

Note: This post is an update and extension to a previous blog post “Running end-to-end Cypress tests for your fullstack CI/CD deployment with Amplify Console“

Overview

Automating the process of building, testing, and deploying software allows for faster delivery of high-quality software and can help reduce the risk of errors, as developers can catch and address issues early on in the development cycle. Cypress is a tool for building and running front-end test suites for end-to-end testing of mobile and web applications, and it can be integrated with AWS Amplify Hosting to provide automated end-to-end testing in a fully-managed CI/CD pipeline. AWS Amplify is a complete solution that lets frontend developers easily build, ship, and host full-stack applications on AWS. But modern full-stack apps can have complex authorization requirements that present challenges when building test cases.

This blog details configuration and strategies using Cypress to test applications built using Amplify and React, and how to run Cypress test suites with AWS Amplify Hosting. We will also cover the use of Parameter Store, a capability of AWS Systems Manager, to secure test user credentials. Finally, we cover using Cypress to test user registration and authenticated user workflows in Amplify apps. This post will guide you through the following steps:

·     Project Setup and Configuration

·     Testing with Authentication and Authorization

·     Continuous Integration with Cypress and Amplify Hosting

·     Cleaning Up

Project Setup and Configuration

The necessary configuration for Amplify apps with Cypress will vary based on the framework used, but generally follow these three steps.

1. Initialize the Amplify application
2. Install Cypress as a development dependency
3. Configure Cypress for end-to-end testing

Initialize the Amplify application

To get started, initialize the Amplify application using step-by-step instructions for your preferred JavaScript framework. Amplify docs have step-by-step instructions for setting up projects for the most popular web frameworks; the following links lead to prescriptive tutorials for configuring an Amplify application from scratch in four of the most popular modern JavaScript frameworks:

• React
• Angular
• Vue.js
• Next.js

Install Cypress as a development dependency

Whether you are using a sample application bootstrapped from the Amplify tutorial docs or an existing application, installing and configuring Cypress should be the next step and can be accomplished with a single command.

$ npm i cypress --save-dev

Configure Cypress for end-to-end testing

The initial run of Cypress will facilitate configuration of the project via a GUI with the option to branch out into a tutorial

$ npx cypress open

Choose End-To-End Testing when prompted. If you are new to creating Cypress test cases, you will be given the option to create and browse a sample test suite that will cover much of the functionality. In the following sections we will explore creating your own test cases against an Amplify front-end application.

Testing with Authentication and Authorization

Amplify makes it easy for developers to enable authorized users to access cloud resources. One of the traditional struggles of building testing strategies is to what extent the tests should rely on backend services. Development environments are often deployed on multi-user environments that can have availability and stability challenges. Application security mechanisms such as authorization services and TLS-terminating reverse proxies are sometimes maintained by an outside team. These challenges frequently lead testers to mock backend dependencies in their end-to-end tests. In the best case, mocks can help with testing stability and accuracy, but they can cause coverage gaps in the application test suite.

AWS customers need to know their backend cloud services are accessed only by authorized users. Amplify Auth integrates with identity providers such as Amazon Cognito to provide the foundation for secure web and mobile apps. Amazon Cognito identity pools allow authenticated users to obtain temporary, scoped-down credentials from AWS Identity and Access Management (IAM) to make direct AWS API calls. Amazon Cognito user pools extend this mechanism to support grouping users and direct support for OAuth 2.0/OIDC flows. In the following sections, we cover best practices to bootstrap Amplify Auth into your end-to-end test cases to invoke AWS services as part of a Cypress test suite. If an application strives to have maximal end-to-end test coverage for security, you should test both the user registration UI flow and those available to authenticated users.
The following code shows a simple React page that requires authentication and a test case to demonstrate the usage of our new customer command.

import React from 'react';
import './App.css';
import { Amplify } from 'aws-amplify';
import { withAuthenticator, Button, Heading } from '@aws-amplify/ui-react';
import '@aws-amplify/ui-react/styles.css';

import awsExports from "./aws-exports";
Amplify.configure(awsExports);

function App({ signOut, user }) {
  return (
    <>
      <Heading>Hello {user.username}</Heading>
      <Button onClick={signOut}>Sign out</Button>
    </>
  );
}

export default withAuthenticator(App, true);

Testing the User Registration UI Flow

The Amplify UI component library for React provides an Authenticator component that supports both authentication and user registration with Amazon Cognito user pools. The following sample Cypress spec tests basic registration of the Amplify Auth component from Amplify UI for React. We will use the Amplify UI components custom data-* attributes to easily locate these elements in the DOM. Using data-* attributes is a best practice that insulates our testing code from changes as the component libraries evolve. Builders using the Amplify UI component libraries, whether with Amplify Studio or a distinct import, can use the `testId` property, which in turn renders in the DOM as a data-testid attribute.

Create the file cypress/e2e/register_spec.cy.js with the following contents to test the Authenticator component.

describe('Registration', function() {
  // Step 1: setup the application state
  beforeEach(()=> {
    cy.visit('/');
  })
  describe('first visit', () => {
     it ('should display login component with sign in UI visible', () => {
      cy.get('div[data-amplify-authenticator]');
      cy.get('form[data-amplify-authenticator-signin]').should('be.visible');
    })
    
    it('should allow registration and prompt for email validation', ()=> {
      cy.get('button').contains('Create Account')
        .click();
      cy.get('form[data-amplify-authenticator-signup]')
        .should('be.visible');
      cy.get('input[name="username"]').type('user'+Date.now());
      cy.get('input[name="password"]').type('Password123!');
      cy.get('input[name="confirm_password"]').type('Password123!');
      cy.get('input[name="email"]').type('foo@bar.com')
      cy.get('button[type="submit"]').click();
      
      cy.get('form[data-amplify-authenticator-confirmsignup]')
        .should('be.visible')
        .children()
        .should('include.text','We Emailed You');
    })
  })
})

While the previous code effectively tests the user interface, registration flows often require multi-stage asynchronous account creation processes to fully initialize the user account. For example, the default Amazon Cognito user registration workflow sends a code to the provided email address and requires it to be submitted to fully activate a user.

Testing Authenticated User Flows

Another option is to pre-register integration test users and then reuse those credentials across the test suite. This introduces the requirement to secure the credentials from unauthorized use and disclosure. Parameter Store, a capability of AWS Systems Manager,  is useful for managing credentials used in these test cases. Parameter Store provides secure, hierarchical storage for configuration data management and secrets management. After manually registering for a user account to use for testing, the configured password can be added to ParameterStore via the AWS CLI.

$ aws ssm put-parameter --name /amplify/testProject/testUser --value {password} --type SecureString

The SecureString parameter type provides encryption at rest along with the ability to use custom keys from the AWS Key Management Service (AWS KMS). Restricting access to Systems Manager parameters using IAM policies enables further fine-grained access control to the credentials as required. Using a prefix of /amplify/ for the name will ensure the parameter can be loaded and decrypted by the Amplify Hosting service’s default IAM policy (as described in the upcoming section on continuous integration).

Developers appreciate the ability to run unit tests within their IDE, and AWS authorization is required to retrieve and decrypt these credentials from Parameter Store. Cypress test suites run inside a web browser context and are subject to the browser sandbox, which limits their ability to read from INI files or access environment variables which are commonly used to store AWS credentials on developer workstations. Cypress configuration, however, runs in a NodeJS context which in turn we would use to access the credentials to use for testing. The function fromNodeProviderChain() in the AWS SDK for JavaScript v3 returns a CredentialProvider object that has a well-defined process for retrieving credentials from a runtime environment; we can use this function in the Cypress configuration.

We first install the necessary AWS SDK for JavaScript modules as development dependencies.

$ npm i @aws-sdk/client-ssm @aws-sdk/credential-providers @cypress/webpack-preprocessor --save-dev

const { defineConfig } = require('cypress');
const { fromNodeProviderChain } = require('@aws-sdk/credential-providers')
const preprocessor = require('@cypress/webpack-preprocessor');

async function resolveAWSCreds(config) {
  config.env.awscredentials = await fromNodeProviderChain()();
  return config;
}

module.exports=defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      on ("file:preprocessor", preprocessor())
      config.baseUrl = 'http://localhost:3000/';
      return resolveAWSCreds(config);
      }
    },
});

The preceding code adds the developer’s credentials to Cypress.env which makes them available to be used by Cypress test suites running in the browser. As we will see later in the blog, using the SDK-provided credential provider chain will also support our CI/CD use cases as well.

At this point we can use the test runner’s AWS credentials to look up the application test credentials from Parameter Store. Rather than go through form-based authentication for every test case, Cypress Best Practices recommend programmatic authentication to avoid the overhead of re-authentication. We can create a custom Cypress command authenticate that we can use to retrieve test credentials and then sign in via a direct call to Amplify Auth. Cypress commands enable reuse of logic across our different test suites.

import { SSMClient, GetParameterCommand } from '@aws-sdk/client-ssm';
 import { Amplify, Auth } from 'aws-amplify';
 import awsmobile from '../../src/aws-exports';
 
 Amplify.configure(awsmobile);
 
 Cypress.Commands.add('authenticate', async (user) => {
   const ssmClient =
     new SSMClient({
       credentials: Cypress.env('awscredentials'),
       region: awsmobile.aws_project_region
     });
 
   const command = new GetParameterCommand ({
     Name: `/amplify/testProject/${user}`,
     WithDecryption: true
   });
   const { Parameter } = await ssmClient.send(command);
   return await Auth.signIn(user, Parameter?.Value);
 })

Once the command is defined, we can use it in a Cypress spec and test our authenticated flows.

describe('Authenticator:', function() {    
  describe('sign in', ()=> {
    it ('should display username greeting', () => {
      cy.authenticate('testUser');
      cy.visit('/');
      cy.get('.amplify-heading').contains('testUser')
    })
  })
});

Figure 1. Successful run of Cypress test spec using the cy.authenticate() custom command.

By default, Cypress Test Isolation will not propagate state such as cookies and local storage between test cases. It’s often desirable to cache our authenticated credentials for each test run to speed up test runs. Since Amplify stores user credentials in browser local storage, authentication will not persist between test cases without additional code. In the following example, we log on at the beginning of the test suite using the before() hook and then cache the localStorage in memory. Before each test case is run, we restore the contents to the window’s localStorage using the beforeEach() hook and the onBeforeLoad callback provided by the Cypress visit() command.

let storageCache;

describe('Authenticated User Cases:', function() {
  before(()=> {
    cy.authenticate('testUser').then((_) => {
      storageCache={...localStorage};
    }
  );
  })
  beforeEach(() => {
    cy.visit('/', {
      onBeforeLoad(win) {
        Object.entries(storageCache).forEach(([k,v]) => {
          win.localStorage.setItem(k,""+v)
        })
      }
    })
  })
  describe('sign in', ()=> {
    it ('should display username greeting', () => {
      cy.get('.amplify-heading').contains('testUser')
    })
  })
  describe('sign in credentials carry over', ()=> {
    it ('should display username greeting', () => {
      cy.get('.amplify-heading').contains('testUser')
    })
  })
});

If you want to extend this method to an entire test suite, you could move the before() and beforeEach() hooks into the Cypress support file, or extend the authenticate command to directly interface with window.localStorage.

Continuous Integration with Cypress and Amplify Hosting

Amplify Hosting provides a git-based workflow for building, deploying, and hosting your full-stack serverless web apps. This enables continuous integration and deployment of both frontend and backend resources when used in conjunction with Git-based repositories like GitHub and AWS CodeCommit. Cypress tests can be configured to run as part of the build and deployment process, so you can confirm all your tests pass before deploying changes to a live environment.
Follow “Getting started” in the Amplify Hosting documentation to set up continuous integration and hosting for your Amplify app. Once you configure Amplify to access your code repository, it will inspect your project’s package.json. When Amplify Hosting detects Cypress in your project dependencies, it will configure some end-to-end testing steps. The build specification is a YAML file that you can edit directly in the Amplify console or download to the root of your code repo (as amplify.yml). The following example file was tested with React 18 and Cypress 12.

frontend:
  phases:
    preBuild:
      - npm install
    build:
      - npx run build
  artifacts:
    baseDirectory: public
    files:
      - '**/*'
  cache:
    paths:
    - node_modules/**/*
test:
  phases:
    preTest:
      commands:
        - npm install pm2 wait-on mocha mochawesome mochawesome-merge mochawesome-report-generator --save-dev
        - npx pm2 start "npm -- start"
        - npx wait-on http-get://localhost:3000
    test:
      commands:
        - 'npx cypress run --reporter mochawesome --reporter-options "reportDir=cypress/report/mochawesome-report,overwrite=false,html=false,json=true,timestamp=mmddyyyy_HHMMss"'
    postTest:
      commands:
        - npx mochawesome-merge cypress/report/mochawesome-report/mochawesome*.json -o cypress/testConfig.json
        - npx marge -o cypress cypress/testConfig.json
        - npx pm2 kill
  artifacts:
    baseDirectory: cypress
    configFilePath: testConfig.json
    files:
      - '**/*.mp4'
      - 'testConfig.html'
      - 'assets/**'

Now you should see your Cypress test suite will be run as part of Amplify Hosting’s deployment process. Notice that the call to Parameter Store works without any further configuration. Peering under the hood, the method fromNodeProviderChain we used in the Cypress config file retrieves the IAM credentials used by the build process. By default, Amplify Hosting uses the AWS managed IAM policy AdministratorAccess-Amplify which allows it to retrieve the parameters indexed by the key /amplify/. Customers who need more fine-grained control around test credentials can update the service role used by Amplify Hosting with a custom policy.

Finally, when test cases fail in a CI/CD environment, developers need to be able to access diagnostic information to troubleshoot. Cypress supports a flexible system for configuring test reporting. In the preceding build specification, we install the mochawesome package and use command-line options to configure it when invoking Cypress. When test runs fail (or succeed), you will be able to retrieve Cypress test results from the Amplify console. Cypress will also generate videos and screenshots of tests that can be downloaded for further analysis using the Download Artifacts button.

Figure 2. A failed Test in Amplify Hosting pipeline will cancel the deployment. The Amplify console provides options to help diagnose the failure.

Figure 3. Amplify console displays information on completed tests, including total duration and an option to download test artifacts.

Cleaning Up

• Delete the application from Amplify
• Delete the parameter from Parameter Store

Conclusion

In this blog, we’ve shown the basics of configuring Cypress to work with Amplify applications, including Amplify Auth and Amplify Hosting modules. We’ve shown how to use Parameter Store to limit disclosure of testing credentials for live apps. And we’ve covered how to implement some best practices to streamline your end-to-end Cypress test suites. Please comment and let us know how this works for you. To get started with Amplify, try it on the AWS Free Tier.

About the author: