Extending AWS CodeBuild with Custom Build Environments
AWS CodeBuild is a fully managed build service that compiles source code, runs tests, and produces software packages that are ready to deploy. CodeBuild provides curated build environments for programming languages and runtimes such as Java, Ruby, Python, Go, Node.js, Android, and Docker. It can be extended through the use of custom build environments to support many more.
Build environments are Docker images that include a complete file system with everything required to build and test your project. To use a custom build environment in a CodeBuild project, you build a container image for your platform that contains your build tools, push it to a Docker container registry such as Amazon EC2 Container Registry (ECR), and reference it in the project configuration. When building your application, CodeBuild will retrieve the Docker image from the container registry specified in the project configuration and use the environment to compile your source code, run your tests, and package your application.
In this post, we’ll create a build environment for PHP applications and walk through the steps to configure CodeBuild to use this environment.
Create the demo resources
To begin, we’ll clone codebuild-images from GitHub. It contains an AWS CloudFormation template that we’ll use to create resources for our demo: a source code repository in AWS CodeCommit and a Docker image repository in Amazon ECR. The repository also includes PHP sample code and tests that we’ll use to demonstrate our custom build environment.
- Clone the Git repository:
- Create the CloudFormation stack using the template.yml file. You can use the CloudFormation console to create the stack or you can use the AWS Command Line Interface:
After the stack has been created, CloudFormation will return two outputs:
- BuildImageRepositoryUri: the URI of the Docker repository that will host our build environment image.
- SourceCodeRepositoryCloneUrl: the clone URL of the Git repository that will host our sample PHP code.
Build and push the Docker image
Docker images are specified using a Dockerfile, which contains the instructions for assembling the image. The Dockerfile included in the PHP build environment contains these instructions:
This Dockerfile inherits all of the instructions from the official PHP Docker image, which installs the PHP runtime. On top of that base image, the build process will install Python, Git, the AWS CLI, and Composer, a dependency management tool for PHP. We’ve installed the AWS CLI and Git as tools we can use during builds. For example, using the AWS CLI, we could trigger a notification from Amazon Simple Notification Service (SNS) when a build is complete or we could use Git to create a new tag to mark a successful build. Finally, the build process cleans up files created by the packaging tools, as recommended in Best practices for writing Dockerfiles.
Next, we’ll build and push the custom build environment.
- Provide authentication details for our registry to the local Docker engine by executing the output of the login helper provided by the AWS CLI:
- Build and push the Docker image. We’ll use the repository URI returned in the CloudFormation stack output (BuildImageRepositoryUri) as the image tag:
After running these commands, your Docker image is pushed into Amazon ECR and ready to build your project.
Configure the Git repository
The repository we cloned includes a small PHP sample that we can use to test our PHP build environment. The sample function converts Roman numerals to Arabic numerals. The repository also includes a sample test to exercise this function. The sample also includes a YAML file called a build spec that contains commands and related settings that CodeBuild uses to run a build:
This build spec configures CodeBuild to run two commands during the build:
composer installto install configured dependencies, such as PHPUnit before the build.
phpunit teststo run the unit tests during the build.
We will push the sample application to the CodeCommit repo created by the CloudFormation stack. You’ll need to grant your IAM user the required level of access to the AWS services required for CodeCommit and you’ll need to configure your Git client with the appropriate credentials. See Setup for HTTPS Users Using Git Credentials in the CodeCommit documentation for detailed steps.
We’re going to initialize a Git repository for our sample, configure our origin, and push the sample to the master branch in CodeCommit.
- Initialize a new Git repository in the sample directory:
- Add and commit the sample files to the repository:
- Configure the git remote and push the sample to it. We’ll use the repository clone URL returned in the CloudFormation stack output (SourceCodeRepositoryCloneUrl) as the remote URL:
Now that our sample application has been pushed into source control and our build environment image has been pushed into our Docker registry, we’re ready to create a CodeBuild project and start our first build.
Configure the CodeBuild project
In this section, we’ll walk through the steps for configuring CodeBuild to use the custom build environment.
- In the AWS Management Console, open the AWS CodeBuild console, and then choose Create project.
- In Project name, type php-demo.
- From Source provider, choose AWS CodeCommit. From Repository, choose codebuild-sample-php.
- In Environment image, select Specify a Docker image. From Custom image type, choose Amazon ECR. From Amazon ECR Repository, choose codebuild/php. From Amazon ECR image, choose latest.
- In Build specification, select Use the buildspec.yml in the source code root directory.
- In Artifacts type, choose No artifacts.
- Choose Continue and then choose Save and Build.
- On the next page, from Branch, choose master and then choose Start build.
CodeBuild will pull the build environment image from Amazon ECR and use it to test our application. CodeBuild will show us the status of each build step, the last 20 lines of log messages generated by the build process, and provide a link to Amazon CloudWatch Logs for more debugging output.
CodeBuild supports a number of platforms and languages out of the box. By using custom build environments, it can be extended to other runtimes. In this post, we built a PHP environment and demonstrated how to use it to test PHP applications.
We’re excited to see how customers extend and use CodeBuild to enable continuous integration and continuous delivery for their applications. Please leave questions or suggestions in the comments or share what you’ve learned extending CodeBuild for your own projects.