Containers

Automatically deploying your container application with AWS Copilot

Taking an application from idea to working implementation that people can interact with is a multistep process. Once the design is locked in and the code is written, the next challenge is how to deploy and deliver the application to users. One way to do this is using a Docker container and a tool like AWS Copilot to automatically provision supporting infrastructure for running the container. If you are not yet familiar with AWS Copilot, you should read more about it in the introduction to AWS Copilot.

Copilot can be used to build and deploy applications from the command line by typing direct commands like copilot svc deploy, however, this isn’t the best way to use copilot in the long term, especially at scale with multiple developers and multiple services. This article will build on the basic usage of Copilot to show how you can start automating application releases. We will start with a basic release pipeline which builds, pushes, and deploys a container automatically each time you push a code change to your source code repository. Then we will implement a best practices pipeline which has multiple stages and tests to ensure that the application is working before it is released to production. Finally, we will walk through a real world scenario of discovering a production bug and releasing a fix to end users.

Meet the application

Imagine you work for a startup called “String Services,” which is aiming to be the premier provider of online string manipulation APIs. Your employer has decided to start offering a string manipulation service called “reverse.” The service will accept any string as input and return a reversed version of the string. It’s your job to deploy this new service to your eager customers. Let’s start with a look at the code, written in Node.js:

var getRawBody = require("raw-body");
var http = require("http");

var server = http.createServer(function (req, res) {
  getRawBody(req)
    .then(function (buf) {
      res.statusCode = 200;
      res.end(buf.toString().split("").reverse().join(""));
    })
    .catch(function (err) {
      res.statusCode = 500;
      res.end(err.message);
    })
});

server.listen(3000);

Accompanying the application code is a simple multistage Dockerfile, which is used to install the application dependencies and then create a minimal Docker image for the application:

FROM node:14 AS build
WORKDIR /srv
RUN npm install raw-body

FROM node:14-slim
WORKDIR /srv
COPY --from=build /srv .
ADD . .
EXPOSE 3000
CMD ["node", "index.js"]

With these two files you have all you need to deploy the string reverser application with Copilot. As covered in the introduction to Copilot, you can use the copilot init command to run a wizard that detects the application and automatically deploys it. At the end of the process, Copilot gives you a URL for the application.

In this case, String Services owns the domain https://string.services and wants to host the application on it so you can use commands like this to deploy the simple string reversing service in two environments using a custom domain:

copilot app init --domain string.services 
copilot env init --name test
copilot env init --name prod
copilot svc init --name reverse
copilot svc deploy --name reverse --env test
copilot svc deploy --name reverse --env prod

At the end, there are two copies of the services in two environments, and you can send a string to the service URL for an environment and get the string back reversed:

$ curl -d "Hello" https://reverse.prod.std.string.services
olleH

Automate application releases

Deploying the application from the CLI with Copilot wasn’t too hard, but String Services is building a massive standard library of different online string manipulation services. They might have 100’s of services eventually, with many of them requiring regular updates. There are a few potential paths:

  • Centralized deployments – All service deploys have to go through a specific person or small group of people who run Copilot to do each service deploy. This separates developers from direct access to deployments, and it also severely bottlenecks the ability to deploy services.
  • Decentralized deployments – You teach all the developers how to use Copilot and make them responsible for deploying and updating their own services. This works, however, developers are all individually pushing changes, and sometimes changes that break the services!
  • Automation with guard rails – The best of both worlds: all developers on the team are empowered to do deployments, however, all the deployments go through a centralized, automated pipeline, which ensures that the service is working properly before the code reaches your customers.

With this third, best of both worlds approach developers can use Copilot if needed, but they can also choose to just git push their code to a repository, which starts an automated process to deploy the code.

You can use a few commands to establish an automated pipeline for the application, which automatically builds and pushes the application on Git push:

copilot pipeline init
git push
copilot pipeline update

With the pipeline deployed, you can start using copilot pipeline status to monitor the pipeline.

You can also look at the pipeline in the AWS CodePipeline service console. At this point, you already have basic automation. A developer can deploy to both the test and production environments just by changing a line of code and then doing a git commit and git push command. Developers actually don’t even need to have Copilot installed anymore or understand how to use it, they just need to know how to use Git.

Add some integration tests

The next step is to add some tests to ensure that the service hasn’t broken when a developer pushes a change. There are multiple types of tests, but one of the most effective types of tests at catching major issues with a service is an integration test. The goal of an integration test is to use the service as a real user would, and verify that the service produces the correct results. In this case, the service is used by sending a string to it’s endpoint, and then the service responds with a reversed copy of the string. You can write an integration test that posts data to the service and then verifies that the returned string is correct.

The developers of the string reverse service have supplied a convenient integration test that can be run against the service:

var superagent = require('superagent');
var expect = require('expect');

const url = process.env.APP_URL;

if (!url) {
  throw new Error('Test process requires that env variable `APP_URL` is set');
}

test('should be able to reverse a simple string', async () => {
  const res = await superagent.post(url).send('Hello');
  expect(res.text).toEqual('olleH');
});

Copilot provides an easy way to add this test script to the pipeline. In the file pipeline.yml, there is a list of the deployment stages with a way to add test commands, so all you have to do is install the test dependencies and call the integration test with its expected environment variable with the deployed test environment’s URL:

# The deployment section defines the order the pipeline will deploy
# to your environments.
stages:
    - # The name of the environment to deploy to.
      name: test
      # Optional: use test commands to validate this stage of your build.
      test_commands:
        - npm install --prefix test
        - APP_URL=https://reverse.test.std.string.services npm test --prefix test
    - # The name of the environment to deploy to.
      name: prod

With the test script added to the pipeline file you can run a couple commands to update the pipeline to have tests:

git push
copilot pipeline update

Now whenever a developer pushes a code change, the code is deployed to the test environment first, the integration tests are run against that test environment, and only if the tests passed the code is deployed to the production environment that customers are using. This shows up as some new steps in the pipeline status:

In the above example, the test commands have succeeded and deployment has moved onto the production, releasing the application update to customers.

Release a bug fix

Once the service is released to customers everything is good for a couple weeks, but then the first customer bug report comes in: “I tried to reverse a string, and got weird question mark characters back!” Fortunately, the customer includes a reproduction case with their bug report:

$ curl -d "Hello ?" https://reverse.prod.std.string.services 
�� olleH

It takes a bit of investigation but the answer becomes clear. The string reverse operation that has been implemented just splits the string on byte boundaries and reverses each byte in the string. This works fine for basic ASCII strings, where each character in the string is represented by a single byte. However, most modern systems implement Unicode. Unicode is a character set that can be added to a string using an encoding technique like UTF-8. This allows extra symbols like emojis that are represented by sequences of two to four consecutive bytes. When the string reverser application reverses these byte sequences it is reversing each individual byte, scrambling the byte sequences into an uninterpretable jumble that renders as question mark symbols.

It’s time to release a bug fix to the service. One of the best ways to ensure that your service continues growing in reliability and stability over time is to treat any customer facing bug as a bug in two places. The first place is in the application code, of course. The second place there is a bug is in the tests. If the tests had adequate coverage, they would have tested this in the first place and caught this problem.

The first step is to add a new test to the test suite and verify that it fails and blocks the deploy pipeline to production:

test('should be able to reverse a string containing UTF-8 characters', async () => {
  const res = await superagent.post(url).send('Hello ?');
  expect(res.text).toEqual('? olleH');
});

After running git push you can see that the pipeline runs through the release process but then blocks with an error, prior to releasing the application to production. When looking at the pipeline in the AWS CodePipeline console, we can see some more information.

If we click on “Details” on the TestCommands stage, it is clear that this failure came from the test that was just added:

This is a good thing! It means that the test is catching the issue. After the issue is fixed you can have confidence that the tests will prevent this bug from ever reoccurring and reaching production again.

Now fixing the issue on the application side is easy. The open source package runes does Unicode-aware string splitting. If we pull this package into the application and use it, we can split the string into substrings which align with the real byte boundaries of the characters they represent.

var getRawBody = require("raw-body");
var runes = require('runes');
var http = require("http");

var server = http.createServer(function (req, res) {
  getRawBody(req)
    .then(function (buf) {
      res.statusCode = 200;
      let stringRunes = runes(buf.toString());
      res.end(stringRunes.reverse().join(""));
    })
    .catch(function (err) {
      res.statusCode = 500;
      res.end(err.message);
    })
});

server.listen(3000);

process.once('SIGTERM', function () {
  server.close();
});

After running git commit and git push again this change is pushed to the pipeline. This time the tests pass and the change is rolled out to production.

Rerunning the customer’s test case verifies that the issue is fixed, and the string reverse service is operating as intended:

$ curl -d "Hello ?" https://reverse.prod.std.string.services
? olleH

Conclusion

This article has covered the steps involved with automating an application deployment using Copilot:

  • First building a basic deploy pipeline
  • Enhancing the pipeline with integration tests on a test environment before code reaches production
  • Fixing an application issue by deploying a new test case, then deploying a bug fix to the application

Hopefully this imaginary scenario helps you the next time you are ready to automate your own container deployments as well! You can find the full sample code for the application and its pipeline on Github. This repository is actually still connected to a Copilot pipeline, so you can open a PR to request a change, and if merged in the code will actually go through the same pipeline and be deployed on the sample service at: https://reverse.prod.std.string.services

TAGS: , , ,
Nathan Peck

Nathan Peck

Developer advocate for serverless containers at Amazon Web Services, working on Amazon Elastic Container Service and AWS Fargate. Likes building startup architecture and microservices.