Integration & Automation

AWS CloudFormation custom resource creation with Python, AWS Lambda, and crhelper

In this post, we’ll cover how to author robust AWS CloudFormation custom resources using AWS Lambda and the custom resource helper (crhelper) framework for Python. Examples of this can be found in the Amazon EKS Architecture, AWS Cloud9 Cloud-Based IDE, and Axomo Quick Starts. For the uninitiated, we’ll quickly go over the key technologies involved in this.

AWS CloudFormation is a service that enables you to describe and provision all the infrastructure resources in your cloud environment. You can model your environment in JSON or YAML templates, or in code, by using tools like the AWS Cloud Development Kit (CDK).

Because AWS CloudFormation provides a powerful extension mechanism through AWS Lambda-backed custom resources, you can write your own resources to extend AWS CloudFormation beyond AWS resources and provision any other resource you can think of. For example, you can integrate a third-party software as a service (SaaS) product, or you can even provision on-premises resources in hybrid environments. Another great use case is to extend AWS CloudFormation by providing utility resources that perform tasks to transform or process properties of your infrastructure.

Custom resources, although powerful, can be daunting. Also, implementing a robust, best-practice resource can take some trial and error, combined with a big chunk of utility code to handle things like signaling status, exception handling, timeouts, etc.

Enter crhelper, an open-source project that assists in writing custom resources by implementing best practices and abstractions to simplify the resource code and ease the burden of implementing some common patterns in custom resources. Let’s dive in and walk through the creation of a custom resource using crhelper.

Create the custom resource

Let’s start by putting together a resource that returns the sum of two numbers. First, make sure you are running a *nix prompt (Linux, Mac, or Windows subsystem for Linux). The shell environment should be configured with Python 3.5 or higher and to the AWS Command Line Interface (AWS CLI).

Create an empty folder that you’ll use to place your Lambda source. Then use pip to install crhelper into the folder, and create the lambda_function.py file to put your resource code into.

mkdir sum_function
cd sum_function
pip install -t . crhelper
# on some systems pip may fail with a distutils error, if you run into this, try running pip with the –system argument
# pip install —system -t . crhelper
touch lambda_function.py

Now open the lambda_function.py file in your favorite editor or integrated development environment (IDE), and place the following code into the lambda_function.py file.

from crhelper import CfnResource

helper = CfnResource()

@helper.create
@helper.update
def sum_2_numbers(event, _):
    s = int(event['ResourceProperties']['No1']) + int(event['ResourceProperties']['No2'])
    helper.Data['Sum'] = s
@helper.delete
def no_op(_, __):
    pass

def handler(event, context):
    helper(event, context)

Looking at the code, we’re first importing and instantiating the crhelper CfnResource class, and then we define our Sum function with the create and update decorators. These decorators mark which function in your code should be invoked for the different CloudFormation stack actions (Create, Update, and Delete). We’ve defined both create and update to the same function, and we have omitted the delete decorator. This results in the Sum function being invoked on both create and update events, while delete events pass without invoking any custom code because, in this case, there are no underlying resources to delete.

For the next step, you’ll need a basic Lambda execution role. Most people who have used Lambda will already have this role created; if not, follow the steps in the Lambda documentation to create one.

Next, package the code up and push your function up to the Lambda service, substituting the role ARN for the ARN of a Lambda basic execution role in your account.

zip -r ../sum.zip ./
aws lambda create-function \
    --function-name "crhelper-sum-resource" \
    --handler "lambda_function.handler" \
    --timeout 900 \
    --zip-file fileb://../sum.zip \
    --runtime python3.7 \
    --role "arn:aws:iam::123412341234:role/lambda-cli-role"

You’ll need to take down the FunctionArn from the output to use in the next step.

Test the custom resource in a template

Now you can create a basic AWS CloudFormation template to test this out.

AWSTemplateFormatVersion: "2010-09-09"
Resources:
  SumResource:
    Type: "Custom::Summer"
    Properties:
      ServiceToken: "arn:aws:lambda:us-west-2:123412341234:function:crhelper-sum-resource"
      No1: 1
      No2: 2
Outputs:
  Sum:
    Value: !GetAtt SumResource.Sum

If you execute this template in AWS CloudFormation (in the same region as the Lambda function), you should see that the outputs contain Sum with a value of 3 as calculated by the Lambda function. Try updating the template by entering an invalid entry (like a string) for one of the numbers, and see how crhelper is able to help surface errors to AWS CloudFormation.

Conclusion

Although this post aims to provide a very basic walkthrough of how to build custom resources using crhelper, the crhelper documentation provides a more complete example of its usage. Or, for a look at a real-world implementation, see the Amazon Elastic Kubernetes Service (Amazon EKS) Quick Start, which uses crhelper for each one of its nine custom resources.