AWS Compute Blog

Continuous Integration/Deployment for AWS Lambda functions with Jenkins and Grunt – Part 1

Daniele Stroppa Daniele Stroppa, AWS Solution Architect

Developing, testing and deploying your AWS Lambda functions can be a tedious process at times: write your function in your preferred editor/IDE, package it with any additional node module, upload it to AWS and test it using the console. Ideally, you would develop and test your function locally, upload it to your repository and let your CI tool deploy it for you.

In this post we’ll show how to make use of tools such as Grunt (a Javascript task runner that can be used to automate tasks such as building and packaging) and the grunt-aws-lambda plugin to execute and test you AWS Lambda functions locally.

 

Create the AWS Resources

Throughout this post we’ll use the CreateThumbnail AWS Lambda function described in the AWS Lambda documentation. Note that you’ll need to install ImageMagick in your local development environment to follow this guide.

Follow Step 1: Create a Lambda Function in the Getting Started guide and create a function called CreateThumbnail and an appropriate IAM Role. Do not worry about the function code for now as we will upload it later on.

Follow Step 1.1: Create Buckets and Upload a Sample Object to create the source and destination buckets (e.g. lambdapics and lambdapicsresized).

 

Setup the development environment

Let’s start by preparing our local development environment.

Install the AWS CLI and Node.js

If you haven’t done so yet, install and configure the AWS CLI, providing your credentials:

# aws configure

Install Node.js following the instructions for your platform of choice. Node.js comes with npm installed so you should have a version of npm.

Create your AWS Lambda function project

Create a directory for your project and create the initial package.json file:

# mkdir create-thumbs-lambda && cd create-thumbs-lambda
# npm init

Edit the package.json file, add any dependecies for your Lambda function (e.g. the GraphicsMagick and Async modules) and add the AWS SDK, grunt and grunt-aws-lambda to the devDependencies. Your package.json should look something like the following:

{
   "name": "create-thumbs-lambda",
   "version": "0.0.1",
   "description": "AWS Lambda function to create an image thumbnail",
   "main": "CreateThumbnail.js",
   "dependencies": {
      "gm": "^1.17.0",
      "async": "^0.9.0"
   },
   "devDependencies": {
      "aws-sdk": "latest",
      "grunt": "0.4.*",
      "grunt-aws-lambda": "0.8.0",
     "npm": "^2.8.3"
   },
   "scripts": {
      "test": "echo \"Error: no test specified\" && exit 1"
   },
   "author": "Amazon Web Services Inc.",
   "license": "Apache 2.0"
}

Finally, install the required Node.js modules locally:

# sudo npm install

 

Develop, Test, Repeat

Now let’s create the Lambda function. Copy the code from Step 2.1: Create a Lambda Function Deployment Package into a new file. Edit the Lambda function so that the last callback in the async.waterfall section will look like this:

function (err) {
   if (err) {
      msg = 'Unable to resize ' + srcBucket + '/' + srcKey +
      ' and upload to ' + dstBucket + '/' + dstKey +
      ' due to an error: ' + err;
   } else {
      msg = 'Successfully resized ' + srcBucket + '/' + srcKey +
      ' and uploaded to ' + dstBucket + '/' + dstKey;
   }
context.done(err, msg);
}

This is to avoid the grunt-aws-lambda plugin to report a failure even if the function completed successfully. Save your Lambda function as CreateThumbnail.js.

Next, create a file named Gruntfile.js with the following content:

var grunt = require('grunt');
grunt.loadNpmTasks('grunt-aws-lambda');

grunt.initConfig({
   lambda_invoke: {
      default: {
         options: {
            file_name: 'CreateThumbnail.js'
         }
      }
   },
   lambda_deploy: {
      default: {
         function: 'CreateThumbnail'
      }
   },
   lambda_package: {
      default: {
   }
   }
});

grunt.registerTask('deploy', ['lambda_package', 'lambda_deploy'])

The grunt-aws-lambda plugin requires a event.json file containing the event that will trigger the Lambda function execution. The event.json file will look like this:

{
   "Records":[
      {
         "eventVersion":"2.0",
         "eventSource":"aws:s3",
         "awsRegion":"us-east-1",
         "eventTime":"1970-01-01T00:00:00.000Z",
         "eventName":"ObjectCreated:Put",
         "userIdentity":{
            "principalId":"AIDAJDPLRKLG7UEXAMPLE"
         },
         "requestParameters":{
            "sourceIPAddress":"127.0.0.1"
         },
         "responseElements":{
            "x-amz-request-id":"C3D13FE58DE4C810",
            "x-amz-id-2":"FMyUVURIY8/IgAtTv8xRjskZQpcIZ9KG4V5Wp6S7S/JRWeUWerMUE5JgHvANOjpD"
         },
         "s3":{
            "s3SchemaVersion":"1.0",
            "configurationId":"testConfigRule",
            "bucket":{
               "name":"lambdapics",
               "ownerIdentity":{
                  "principalId":"A3NL1KOZZKExample"
               },
               "arn":"arn:aws:s3:::lambdapics"
            },
            "object":{
               "key":"amazon-web-services-lambda.jpg",
               "size":17565,
               "eTag":"59103e49494baafabe627ce8476c9a3e",
               "versionId":"096fKKXTRTtl3on89fVO.nfljtsv6qko"
            }
         }
      }
   ]
}

Let’s test our lambda function, making sure that it runs with no errors before uploading:

# grunt lambda_invoke

If everything ran without any error, you’ll see a message similar to this:

Message
-------
Successfully resized lambdapics/amazon-web-services-lambda.jpg and uploaded to lambdapicsresized/resized-amazon-web-services-lambda.jpg

Done, without errors.

Once your function is executing without errors, you can package it and upload it with a simple command:

# grunt deploy

 

Conclusion

In this walkthrough we demonstrated how to use Grunt and the grunt-aws-lambda plugin to run your Lambda function in your local environment. In the second part we’ll show how to continuosly integrate and deploy your Lambda function with Jenkins. See the documentation for further information on AWS Lambda.