Developing microservices using container image support for AWS Lambda and AWS CDK

AWS Cloud Development Kit (AWS CDK) is an open source software development framework used to define cloud application resources using familiar programming languages. AWS CDK can build container images locally, deploying them to Amazon Elastic Container Registry (Amazon ECR), and configure them to run as Lambda functions. AWS CDK accelerates onboarding to AWS because there are few new things to learn. CDK enables you to use existing skills and tools, and to apply those to the task of building cloud infrastructure.

AWS Lambda enables packaging and deploying functions as container images of up to 10 GB. This combines the flexibility and familiarity of container tooling with the agility and operational simplicity of Lambda. Customers can use the flexibility and familiarity of container tooling, and the agility and operational simplicity of AWS Lambda to build applications.

Many customers have invested in container tooling, development workflows, and expertise. Without AWS CDK, customers using container tooling and packaging couldn’t get the full benefits of AWS Lambda. They couldn’t use their preferred community or private enterprise container images.

In this blog post, we show how to deploy a serverless HTTP API invoking several Lambda functions packaged as a single container image. The functions use the AWS SDK for JavaScript to retrieve data from an Amazon DynamoDB table. The infrastructure is created and managed with the AWS CDK for TypeScript.

Prerequisites

Deploying this solution requires:

An AWS account with administrator access.
The AWS Command Line Interface (AWS CLI) installed.

Solution

You can use a local development machine to set up an environment or use AWS Cloud9. This post shows examples from the AWS Cloud9 interface. To create an AWS Cloud9 environment, follow the instructions in GitHub.

Open a new terminal in AWS Cloud9 and install jq by running:

sudo yum install jq -y

Clone the GitHub repository containing the sample code:

git clone https://github.com/aws-samples/aws-cdk-lambda-container.git

Create the example dataset

This solution uses the example movies table as a dataset. Refer to the AWS documentation on how to create a DynamoDB Table with the AWS SDK for JavaScript for instructions.

To set up the table, navigate to the Amazon DynamoDB directory in the code repository. Run the create-MoviesTable.sh and load-MovieTable.sh scripts.

cd ~/environment/aws-cdk-lambda-container/DynamoDB 
./create-MoviesTable.sh 
./load-MoviesTable.sh

screenshot of command-line showing script running

Create Lambda functions

Create two Lambda functions: list.js and get.js. The code for these functions is in the GitHub repository.

list.js function: The list function retrieves all movies in the movies table and the code is in GitHub.

get.js function: The get function retrieves an item from the movies table based on two input parameters: the year and title. These parameters are later passed into this function through an HTTP API via Amazon API Gateway. The get.js function code is in GitHub.

Create a Dockerfile

A Dockerfile is a text document that contains the commands a user calls on the command line to assemble a container image. Locate a Dockerfile in the current workspace at: ~/environment/aws-cdk-lambda-container/src/movie-service/Dockerfile.

FROM public.ecr.aws/lambda/nodejs:12 
# Alternatively, you can pull the base image from Docker Hub: amazon/aws-lambda-nodejs:12 
# Copy the Lambda functions 
COPY list.js get.js package.json package-lock.json ${LAMBDA_TASK_ROOT}/ 
# Install NPM dependencies for functions 
RUN npm install

This Dockerfile specifies the publicly available AWS base image for Lambda with Node.js 12: public.ecr.aws/lambda/nodejs:12. It copies the list.js, get.js, package.json, and package-lock.json files into the ${LAMBDA_TASK_ROOT} directory. Then it runs npm install to install the function’s dependencies. The ${LAMBDA_TASK_ROOT} represents the path to our Lambda functions as documented in the AWS documentation on using AWS Lambda environment variables.

Build the container image

Using the Dockerfile and the two Lambda functions, build the container image. The container image includes everything needed to run the Lambda function handler: the function code and dependencies, the runtime binaries, and the operating system inherited from the base image.

From the terminal, run the following commands:

cd ~/environment/aws-cdk-lambda-container/src/movie-service docker 
build -t movie-service . 
docker images | grep movie-service

This is the output:

screenshot of output from running cd ~/environment/aws-cdk-lambda-container/src/movie-service docker build -t movie-service . docker images | grep movie-service

Testing Lambda functions locally

To test the packaged Lambda functions locally, use the AWS Lambda Runtime Interface Emulator (RIE). Lambda Runtime Interface Emulator is a lightweight web server that converts HTTP requests into JSON events to pass to the Lambda functions in the container image.

In the container image, you must configure the following environment variables:

AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, and AWS_REGION for authentication with the AWS SDK. Use the AWS CLI to get the current environment’s credentials and pass those along to the local container using aws configure get.
DYNAMODB_TABLE to pass the Movies table to the Lambda functions.

1. Run the list.js function: This command runs the movie-service image as a container and starts an endpoint for the list.js function locally at http://localhost:9080/2015-03-31/functions/function/invocations:

docker run \ --env DYNAMODB_TABLE=Movies \ 
 --env AWS_ACCESS_KEY_ID="$(aws configure get default.aws_access_key_id)" \ 
 --env AWS_SECRET_ACCESS_KEY="$(aws configure get default.aws_secret_access_key)" \ 
 --env AWS_SESSION_TOKEN="$(aws configure get default.aws_session_token)" \ 
 --env AWS_REGION="$(aws configure get default.region)" \ 
 -p 9080:8080 \ 
 movie-service list.list

2. Open a new terminal to run subsequent commands.
3. Invoke the list.js function:

curl -s "http://localhost:9080/2015-03-31/functions/function/invocations" -d '{}' | jq

4. Run the get.js function: The following command runs the movie-service image as a container and starts up an endpoint for the get.js function locally at http://localhost:9080/2015-03-31/functions/function/invocations:

docker rm -f $(docker ps -q)
docker run \
	--env DYNAMODB_TABLE=Movies \
	--env AWS_ACCESS_KEY_ID="$(aws configure get default.aws_access_key_id)" \
	--env AWS_SECRET_ACCESS_KEY="$(aws configure get default.aws_secret_access_key)" \
	--env AWS_SESSION_TOKEN="$(aws configure get default.aws_session_token)" \
	--env AWS_REGION="$(aws configure get default.region)" \
	-p 9080:8080 \
	movie-service get.get

5. Test the get.js function. This command invokes the get.js function with two variables—year=”2013” and title=”Rush”—to simulate an incoming API Gateway request:

curl -s "http://localhost:9080/2015-03-31/functions/function/invocations" -d '{"pathParameters": {"year": "2013", "title": "Rush"} }' | jq

output screenshot

Deploy the application

Using the local environment, open a new terminal and run:

cd ~/environment/aws-cdk-lambda-container/cdk
npm install

This installs the latest AWS CDK modules in the node_modules directory.

Creating AWS resources using CDK

AWS CDK deploys the architecture via a single CDK stack written in TypeScript. In the cdk/lib directory, open the http-api-aws-lambda-container-stack.ts file and explore the following different CDK constructs.

DynamoDB table

Import the existing DynamoDB table using AWS CDK (as explained in the documentation):

const table = dynamodb.Table.fromTableName(this, 'MoviesTable', 'Movies');

Lambda functions

Create two Lambda functions using AWS CDK DockerImageFunction class. The code attribute is using the static fromImageAsset(directory, props?) method of the DockerImageCode class. It uses the Dockerfile in the src/movie-service directory.

listMovieFunction:

const listMovieFunction = new lambda.DockerImageFunction(this, 'listMovieFunction', {
    functionName: 'listMovieFunction',
    code: lambda.DockerImageCode.fromImageAsset(path.join(__dirname, '../../src/movie-service'), {
    cmd: [ "list.list" ],
    entrypoint: ["/lambda-entrypoint.sh"],
    }),
    environment: {
            DYNAMODB_TABLE: this.table.tableName
    },
});

getMovieFunction:

const getMovieFunction = new lambda.DockerImageFunction(this, 'getMovieFunction',{
    functionName: 'getMovieFunction',
    code: lambda.DockerImageCode.fromImageAsset(path.join(__dirname, '../../src/movie-service'), {
    cmd: [ "get.get" ],
    entrypoint: ["/lambda-entrypoint.sh"],
    }),
    environment: {
            DYNAMODB_TABLE: this.table.tableName
    },
});

Lambda proxy integrations

Amazon API Gateway Lambda proxy integration allows the client to invoke a Lambda function. When a client submits an API request, API Gateway passes the raw request to the integrated Lambda function.

Create two Lambda proxy integrations for the two Lambda functions using LambdaProxyIntegration class, which takes LambdaProxyIntegrationProps as an argument.

listMovieFunctionIntegration:

const listMovieFunctionIntegration = new apigintegration.LambdaProxyIntegration({
        handler: listMovieFunction,
});

getMovieFunctionIntegration:

const getMovieFunctionIntegration =  new apigintegration.LambdaProxyIntegration({
  	handler: getMovieFunction,
});

HTTP API

HTTP APIs for Amazon API Gateway enables developers to create RESTful APIs with lower latency and lower cost than REST APIs. (For more information about HTTP APIs, read Building faster, lower cost, better APIs – HTTP APIs now generally available). HTTP APIs can be used to send requests to Lambda functions.

Create an HTTP API that integrates with the two Lambda functions on the backend. When a client calls this API, API Gateway sends the request to the Lambda function and returns the function’s response back to the client. Here is the code for the HTTP API with a default stage.

const httpApi = new apig.HttpApi(this, "httpApi", { 
	apiName: "httpApi", 
	createDefaultStage: true, 
});

HTTP API routes

HTTP API routes consist of two parts: an HTTP method and a resource path. Routes direct incoming API requests to backend resource, such as AWS Lambda functions.

Create a GET /list route to integrate with the listMovieFunction Lambda function and a GET /{year}/{title} route to integrate with the getMovieFunction Lambda function. For additional details, refer to the HttpRoute class documentation.

httpApi.addRoutes({
  integration: listMovieFunctionIntegration, 
  methods: [apig.HttpMethod.GET], 
  path: '/list',
});

httpApi.addRoutes({
  integration: getMovieFunctionIntegration,
  methods: [apig.HttpMethod.GET],
  path: '/get/{year}/{title}',
});

Provisioning AWS resources using AWS CDK

1. Compile the TypeScript into a CDK program:

cd ~/environment/aws-cdk-lambda-container/cdk
npm run build

2. To create the initial CDK infrastructure in a specified Region (us-east-1 in this example), run the cdk bootstrap command:

export AWS_REGION=us-east-1
cdk bootstrap

AWS CDK uses the same supporting infrastructure for all projects within a Region. The bootstrap command must be run only once in any Region where you create CDK stacks.

3. Deploy the stack using this command:

cdk deploy

(Enter y in response to Do you wish to deploy all these changes (y/n)?).

Note: If you receive an error, check package.json and confirm that all CDK libraries have the same version number (with no leading caret ^). Many CDK project errors stem from mismatched versions. If necessary, correct the version numbers, delete the package-lock.json file and node_modules tree, and run npm install.
The syntax and additional details of these commands are in the documentation.

output with question "Do you wish to deploy all these changes (y/n)"

Testing the HTTP API

Note the HTTP API endpoint of the list Lambda function in the Outputs section: HttpApiAwsLambdaContainerStack.HttpApiendpointlistMovieFunction.

Test the API endpoints:

curl -s https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/list | jq
curl -s https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/get/2013/Rush | jq

The output shows the integration of the HTTP API with the Lambda function in the API Gateway console.

The API Gateway console shows the HTTP API integration with the Lambda function.

Cleanup

To clean up the resources created by AWS CDK, run:

cd ~/environment/aws-cdk-lambda-container/cdk
cdk destroy

(Enter y in response to Are you sure you want to delete (y/n)?).

To remove the Movies DynamoDB table, run:

aws dynamodb --region us-east-1 delete-table --table-name Movies

Conclusion

This post shows how to use AWS CDK and AWS Lambda to build and deploy serverless applications. Building and deploying functions using Infrastructure as Code (IaC) helps reduce manual steps in the AWS Management Console or AWS CLI.

To learn more about AWS CDK, visit the CDKWorkshop site.