AWS DevOps Blog

Building Continuous Deployment on AWS with AWS CodePipeline, Jenkins and AWS Elastic Beanstalk

The following is a guest post by Hubert Cheung, Solutions Architect.

In a deployment pipeline, automation lets you focus on your application and worry less about the manual aspects of deployment or the steps required to get application code updated. In addition to continuously integrating code changes and offering faster feedback to the developers, the setup offers rapid deployment capabilities to significantly reduce lag in the concept to cash value stream map.

In this blog post, we’ll show you how to use GitHub, AWS CodePipeline, Jenkins, and AWS Elastic Beanstalk to create a deployment pipeline for a web application that is updated automatically every time you change your code. For this, we assume you have working knowledge of Linux, Git, and Amazon EC2 as we will cover deploying a Java application within a Linux environment.

Before we get started, let’s cover what CodePipeline, Elastic Beanstalk, and Jenkins are, and the roles the play.

CodePipeline is a continuous delivery service for fast and reliable application updates. It allows you to define, model, and automate your entire release process from your initial commit all the way to production. In this blog, we will detail how to use CodePipeline to deploy a simple Hello World application, but as applications become larger and more complex, the need for automation becomes increasingly important. CodePipeline not only automates your release workflow, but also gives you a visual representation of the state of each release. You get immediate feedback on which stages failed so you can make decisions or revisions without investigating every step, thus reducing the time it takes for code to reach production.

Elastic Beanstalk is an easy-to-use service for deploying and scaling web applications and services developed with Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker. Elastic Beanstalk allows you to focus on development and not on infrastructure, as it will automatically handle provisioning and operating the infrastructure required to get your apps up and running.

Jenkins is a popular continuous integration and continuous delivery tool. Jenkins can build and test your software projects continuously while offering various delivery options as well as a very extensible interface powered by Jenkins plugins. Because the AWS CodePipieline plugin for Jenkins was recently added to the Jenkins plugin repo, it’s only natural that we highlight it in a blog post! When all is said and done, you will have CodePipeline monitoring GitHub, passing source code to Jenkins to build, and deploying to AWS Elastic Beanstalk on each commit.

Set Up GitHub Repo

First, create a GitHub repo. Then, if you already have a Maven application to deploy, feel free to use it. If not, download the binary tar.gz archive and install maven in your development environment. The following commands are sample commands and will differ depending on the version number contained in the filename of the maven archive. For example, these commands would extract the archive to /opt/apache-maven-3.3.3, but if the version downloaded is 3.3.9, it would extract the archive to /opt/apache-maven-3.3.9/.

wget http://mirror.example.com/pub/apache/maven/maven-3/3.3.3/binaries/apache-maven-3.3.3-bin.tar.gz 
sudo tar xzvf apache-maven-3.3.3-bin.tar.gz -C /opt/
rm apache-maven-3.3.3-bin.tar.gz
export PATH=$PATH:/opt/apache-maven-3.3.3/bin
mvn archetype:generate -DgroupId=com.ebdemo -DartifactId=ebdemo -DarchetypeArtifactId=maven-archetype-webapp -DinteractiveMode=false

This will generate a pom.xml file and some source code in an ebdemo folder to display Hello World! After that’s done, add, commit, and push your code up to GitHub.

Set Up the Deployment Environment

This environment is where CodePipeline will deploy our code. Please note that you need to create the Elastic Beanstalk environment in a region that supports CodePipeline.

1. Create an Elastic Beanstalk application named EBDemo.

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.deployment.newapp.html

 

2. Create an Elastic Beanstalk environment named EBDemo-env with the following parameters:

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.environments.html

a.  For Web Server Environment, choose Create web server.

b. For Predefined configuration, choose Tomcat.

c. For Environment type, choose Load Balancing, auto scaling.


d. For Source, choose Sample application.

e. For Environment name, type EBDemo-env.

f. For Environment URL, enter a URL that is specific to your environment. For example, we’ve named ours ebdemo-env-unique.

g. Leave the boxes cleared unless you want to launch into a VPC.

h. If you like, you can choose a key pair. You will use this key pair to connect to your Elastic Beanstalk instances by using SSH.

i.  Although tags are optional, the AWS best practice is to tag everything to provide better visibility into your resources. We’ll tag this environment with an EnvType of Beta.

j. If the instance profile and service role have not been created yet, create and then select them.

k. Click Next and Launch on the next page. Elastic Beanstalk will begin provisioning your environment

Set up the Build Environment:

First, we’ll set up a new Jenkins EC2 instance. To do this, we’ll deploy Jenkins onto a standalone EC2 instance running Amazon Linux. We’ll also need to authorize ports 80 and 22 to access your Jenkins server.

In order for the CodePipeline Jenkins plugin to access CodePipeline, you have a few options:

  • Access/Secret Keys specified in the Jenkins console
  • Environment Variables
  • Java System Properties
  • Credentials Profile File
  • EC2 Instance Profile

As a best practice, we recommend launching your Jenkins server with an IAM role to grant the required permissions to interact with CodePipeline. Here’s how:

1. Sign in to the AWS Management Console and open the IAM console.

2. In the IAM console, choose Roles, then choose Create New Role:

 

3. On the Set Role Name page, type the name of the role. We’ll name it JenkinsAccess.

4. On the Select Role Type page, select Amazon EC2.

5. On the Attach Policy page, filter for and select AWSCodePipelineCustomActionAccess.

6. On the Review page, click Create Role.

The next step is to launch the EC2 instance that will host Jenkins. Launch an instance that:

  1. Runs the Amazon Linux AMI.
  2. Has access to the internet (either through a public or elastic IP in a public subnet, or through a NAT in a private subnet).
  3. Uses the IAM Role you just created.
  4. Can be accessed through port 80 and 22. (As a best practice, lock down the security group to your IP address only.)

For more information, see Launching an Instance.

Installing Maven

In the same way as the development environment, download the binary tar.gz archive and install maven in your Jenkins build server.

wget http://mirror.cogentco.com/pub/apache/maven/maven-3/3.3.3/binaries/apache-maven-3.3.3-bin.tar.gz 
sudo tar xzvf apache-maven-3.3.3-bin.tar.gz -C /opt/ 
rm apache-maven-3.3.3-bin.tar.gz

Maven is now installed at /opt/apache-maven-3.3.3/ in this example.

Set Up Jenkins

To install Jenkins, follow these instructions to add a Yum repo that contains the Jenkins binaries and then install it using yum:

https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+on+Red+Hat+distributions

sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo
sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key
sudo yum install -y jenkins

To confirm Jenkins was installed, run this command:

sudo service jenkins status

If Jenkins is not running, run this command to start it:

sudo service Jenkins start

Jenkins has a default port of TCP/8080, but we’ll use iptables to redirect port 80 to port 8080 and allow local connections.

sudo iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080
sudo iptables -t nat -I OUTPUT -p tcp -o lo --dport 80 -j REDIRECT --to-ports 8080

You can check that the rules were successfully added by running:

sudo iptables -L -t nat

 

 

Open a browser to access the Jenkins portal. Navigate directly to the IP/DNS name of your instance. In the earlier step, we routed traffic on port 80 to redirect to port 8080. At this point, Jenkins is up and running and can be accessed on port 80. That means anyone can access it on port 80 if the security group allows it.

Configure Security in Jenkins

Securing Jenkins is optional, but highly recommended. We’ll set up basic security.

1. From the Jenkins landing page, choose Manage Jenkins.

2. Choose Configure Global Security.

3. Check Enable Security.

4. Under Security Realm, choose Jenkin’s own user database and Allow users to sign up.

5. Under Authorization, choose Matrix-based Security.

6. To the User/group table, type ebdemo, and then choose Add. Use the icon to give the user all permissions. This gives permissions to the account you’ll create in the next two steps.

7. Choose Save.

8. Choose Create an account.

9. Create an account with a Username of ebdemo and a password of your choice.

10. Sign in with your ebdemo account.

 

Install the AWS CodePipeline Plugin

The CodePipeline plugin is now included in the Jenkins plugin repo.

1. From the Jenkins landing page, choose Manage Jenkins.

2. Choose Manage Plugins.

3. On the Available tab, filter for CodePipeline.

4. Choose the Download now and Install after restart button.

5. Check the Restart Jenkins when Installation is complete and no jobs are running box for Jenkins to automatically restart and install the plugin.

Configure Maven in Jenkins

1. On the Jenkins landing page, choose Manage Jenkins.

2. Choose Configure System.

3. Scroll to the Maven (not Maven Configuration) section.

4. Choose the Add Maven button.

5. For Name, type Maven. Be sure the Install Automatically box is cleared.

6. Type the directory where you downloaded and extracted Maven earlier.

7. Choose Save.

Create a Jenkins Build Job

We can now create a Jenkins build job that will poll CodePipeline for new artifacts, build them, and pass the completed build back to CodePipeline.

1. In the Jenkins console, choose New Item.

2. Choose Maven Project.

3. Type a name for the project. For the purpose of this post, we will use ebdemo.

4. Choose OK.

5. On the configuration page for the job, under Source Code Management, choose AWS CodePipeline.

6. Under Build Triggers, select Poll SCM.

7. In the Schedule text box, type * * * * *. This will poll CodePipeline every minute for changes.

8. In the Post-build Actions section, choose Add post-build action, and then choose AWS CodePipeline Publisher.

9. For Location, type the location of the built artifact. In this case, it is target/ebdemo.

10. Choose Save.

The following page should appear:

Jenkins should now be set up to listen for changes coming from CodePipeline.

Create and Configure AWS CodePipeline

1. Sign in to the AWS Management Console and then open the CodePipeline console. Choose Get Started.

2. On the Create pipeline page, type a name your pipeline. For this post, we’ll name it EBDemo.

3. For Source provider, choose GitHub.

4. Choose Connect to GitHub, and then type your GitHub credentials.

5. For Build provider, choose Add Jenkins.

6. The provider name and project name must match the values we used to configure Jenkins:

  • Provider Name: EBDemoJenkins
  • Server URL: <The URL of your Jenkins EC2 instance>
  • Project Name: ebdemo

7. For Deployment provider, choose AWS Elastic Beanstalk. For Application name, type EBDemo. For Environment name, type ebdemo-env.

8. Under AWS Service Role, choose the AWS CodePipeline service role or create one.

9. Review your pipeline, and then choose Go. You should see this:

In about a minute, CodePipeline will take your most recent revision from GitHub, pass it to your Jenkins EC2 instance, and then send the built code to Elastic Beanstalk. Once you see that CodePipeline has completed execution as pictured below, go to the Elastic Beanstalk console to take a look at your environment.

 

You now have a pipeline that will automatically build and deploy to Elastic Beanstalk each time you check in code. Happy coding!