AWS Compute Blog

Continuous Integration/Deployment for AWS Lambda functions with Jenkins and Grunt – Part 2

by Vyom Nagrani | on | in AWS Lambda | | Comments

Daniele Stroppa Daniele Stroppa, AWS Solution Architect

In a previous post we showed how to make use of tools such as Grunt (a Javascript task runner that can be used to automate tasks such as building and packaging) and the grunt-aws-lambda plugin to execute and test your Lambda function in your local environment. In this post we’ll take a step further and show how to use Jenkins to streamline the AWS Lambda functions deployment workflow.

 

Setup the build environment

For our build environment we’ll launch an Amazon EC2 instance using the Amazon Linux AMI and install and configure the required packages. Make sure that the security group you select for your instance allows traffic on ports TCP/22 and TCP/80 and that the IAM role you select for your EC2 instance allows the GetFunction, CreateFunction, UpdateFunctionCode and UpdateFunctionConfiguration Lambda actions and the IAM PassRole action to be able to deploy the Lambda function, e.g.:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1432812345671",
            "Effect": "Allow",
            "Action": [
                "lambda:GetFunction",
                "lambda:CreateFunction",
                "lambda:UpdateFunctionCode",
                "lambda:UpdateFunctionConfiguration"
            ],
            "Resource": [
                "arn:aws:lambda:us-east-1:123456789012:function:CreateThumbnail"
            ]
        },
        {
            "Sid": "Stmt14328112345672",
            "Effect": "Allow",
            "Action": [
                "iam:Passrole"
            ],
            "Resource": [
                "arn:aws:iam::123456789012:role/lambda_exec_role"
            ]
        }
    ]
}

 

Install and configure Jenkins, Git and Nginx

Connect to your instance using your private key and switch to the root user.
First, let’s update the repositories and install Nginx and Git.

# yum update -y
# yum install -y nginx git

To install Jenkins on Amazon Linux, we need to add the Jenkins repository and install Jenkins from there.

# wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo
# rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key
# yum install -y jenkins

As Jenkins typically uses port TCP/8080, we’ll configure Nginx as a proxy. Edit the Nginx config file (/etc/nginx/nginx.conf) and change the server configuration to look like this:

server {
    listen       80;
    server_name  _;

    location / {
            proxy_pass http://127.0.0.1:8080;
    }
}

Start the Jenkins and Nginx services and make sure they will be running after a reboot:

# service jenkins start
# service nginx start
# chkconfig jenkins on
# chkconfig nginx on

Point your browser to the public DNS name of your EC2 instance (e.g. ec2-54-163-4-211.compute-1.amazonaws.com) and you should be able to see the Jenkins home page:

Jenkins Home Page

The Jenkins installation is currently accessible through the Internet without any form of authentication. Before proceeding to the next step, let’s secure Jenkins. Select Manage Jenkins on the Jenkins home page, click Configure Global Security and then enable Jenkins security by selecting the Enable Security checkbox.

For the purpose of this walkthrough, select Jenkins’s Own User Database under Security realm and make sure to select the Allow users to sign up checkbox. Under Authorization, select Matrix-based security. Add a user (e.g. admin) and provide necessary privileges to this user.

Configure Global Security

After that’s complete, save your changes. Now you will be asked to provide a username and password for the user to login. Click on Create an account, provide your username – i.e. admin – and fill in the user details. Now you will be able to log in securely to Jenkins.

 

Install and configure the Jenkins plugins

The last step in setting up our build environment is to install and configure the Jenkins plugins required to deploy your Lambda function. We’ll also need a plugin to interact with the code repository of our choice, GitHub in our case.

From the Jenkins dashboard select Manage Jenkins and click Manage Plugins. On the Available tab, search for and select the following plugins:

Then click the Install button. After the plugin installation is completed, select Manage Jenkins from the Jenkins dashboard and click Configure System. Look for the NodeJS section click the Add NodeJS button. Add a new Node.js version, specify a name (e.g. Node.js 0.10.33) and select Install automatically. Click the Add Installer button and select Install from nodejs.org. Select the version of Node.js that you want to install (e.g. 0.10.33) and add grunt@0.4.* in the Global npm packages to install box. Click Save to confirm your changes.

Jenkins Node JS

Note that it is recommended to install the same version of Node.js that is supported by Lambda.

 

Configure the AWS CLI

Now we are ready to setup and configure the AWS Command Line Interface (CLI).
Make sure that Jenkins is able to use the AWS CLI. Switch to the jenkins user and configure the AWS CLI, providing your credentials:

# aws configure

Note that you might have to change the jenkins user shell from /bin/false to /bin/bash to be able to login successfully.

 

Configure the Jenkins build

On the Jenkins dashboard, click on New Item, select the Freestyle project job, add a name for the job, and click OK. Configure the Jenkins job:

  • Under GitHub Project add the path of your GitHub repository – e.g. https://github.com/awslabs/create-thumbnails-lambda. In addition to the function source code, the repository contains the Gruntfile.js and event.json as explained at the first part of this walkthrough.

Github AWSLabs Project

  • Under Source Code Management provide the Repository URL for Git, e.g.https://github.com/awslabs/create-thumbnails-lambda, and your Github credentials.

Source Code Management

  • In the Build Triggers section, select Build when a change is pushed to GitHub
  • In the Build Environment section, select Provide Node & npm bin/ folder to PATH and choose your Node installation
  • In the Build section, add a Execute Shell step and add the commands to install the required packages and to create the Lambda function bundle:
# npm install
# grunt lambda_package
  • Add a AWS Lambda deployment step, fill in your AWS credentials, your AWS Region (e.g. us-east-1), the Lambda function name (e.g. CreateThumbnail) and change the Update Mode to Code to only update your Lambda function code. Set the Artifact Location to dist/create-thumbs-lambda_0-0-1_latest.zip as this is where Grunt generates the Lambda function bundle

Deploy Lambda Function

To trigger the build process on Jenkins upon pushing to the GitHub repository we need to configure a service hook on GitHub. Go to the GitHub repository settings page, select Webhooks and Services and add a service hook for Jenkins (GitHub plugin). Add the Jenkins hook url: http://:@/github-webhook/.

Jenkins Github Plugin

Now we have configured a Jenkins job in such a way that whenever a change is committed to GitHub repository it will trigger the build process on Jenkins.

 

Let’s kick things off

From your local repository, push the application code to GitHub:

# git add *
# git commit -m "Kicking off Jenkins build"
# git push origin master

This will trigger the Jenkins job. After the job is completed, upload an image (e.g. sample.jpg) to your input S3 bucket (e.g. lambdapics) and verify that a thumbnail version of your image exists in the output S3 bucket (e.g. lambdapicsresized/resized-sample.jpg)

 

Conclusion

In this two-parts walkthrough we demonstrated how to use Grunt and the grunt-aws-lambda plugin to run your Lambda function in your local environment and how Jenkins can help to automate the deployment of a Lambda function. See the documentation for further information on AWS Lambda.