AWS Cloud Operations & Migrations Blog

Looking up information on AWS CloudFormation stack parameters using AWS Lambda

By Jeff Levine, AWS Solutions Architect

AWS CloudFormation provides a common language for you to describe and provision all of the infrastructure resources in your cloud environment. You can specify run time parameters to customize your CloudFormation template’s operation. AWS provides some AWS-specific parameters types to make things easier. For example, if you use the parameter type AWS::EC2::Subnet::Id, CloudFormation will provide a list of the Amazon VPC subnets in the current Region of your AWS account from which you can make a selection. You can access the value of the parameter as a string using the Ref intrinsic function.

Getting more than a string value for a parameter
Let’s say you want to select subnet-12345 from the list, but you’d like to know that the subnet is associated with the Availability Zone us-west-2a so you can use that value in provisioning other infrastructure in the same Availability Zone. For resources created by CloudFormation, you can use the GetAtt intrinsic function to look up information. For stack parameters, you need a different approach. CloudFormation provides a way we can address this challenge using custom resources. In this blog post, I will show how to use custom CloudFormation resources to look up information on stack parameters such as subnets and virtual private clouds (VPCs). You can then use the values elsewhere in your template to help with other provisioning tasks.

CloudFormation custom resources overview
Custom resources enable you to extend the provisioning capabilities of CloudFormation to resources types that you define. Here are the things you need to do to create and use a custom resource.

  1. Define a resource provider that will handle the provisioning of the resource. We will use an AWS Lambda function, a serverless compute task, to handle this. CloudFormation will call this Lambda function to look up information about a subnet or VPC that we specify as a resource property. We will walk through the key parts of the function later in the post.
  2. Define an AWS Identity and Access Management (IAM) role for AWS Lambda execution with permissions that are required to perform the provisioning task. Our function will look up information for VPCs and subnets so our IAM role will need to include both the ec2:DescribeVpcs and ec2:DescribeSubnet permissions. I also recommend including the permissions for Amazon CloudWatch Logs so your Lambda function can log status or error messages.
  3. Define the CloudFormation resource itself. The resource properties will be the AWS Lambda function from step (1) and the subnet or VPC stack parameter (e.g., subnet-12345 or vpc-67890).
  4. CloudFormation invokes the Lambda function to provision the resource. The function looks up the VPC and subnet information and stores them as return values to the function.
  5. You can then use the GetAtt intrinsic function to access the values.

Template walkthrough
I’ve provided a CloudFormation template in GitHub that shows you how to use a custom resource to look up subnet and VPC information. My template provides similar functionality to the GetAtt intrinsic but on stack parameters, so I’ve named my function “GetAttFromParam.”  The template is written in YAML. It also contains the Lambda function, which is written in Python. If you choose to edit the code, pay close attention to the spaces since spacing and indentation play an important role in both languages.

Parameter definition

Our CloudFormation template contains two stack parameters, SubnetName and VPCName. Both of these make use of the parameter types specific to AWS. Here are the definitions:

SubnetName:
  Type: AWS::EC2::Subnet::Id
  Description: Subnet Identifier

VpcName:
  Type: AWS::EC2::VPC::Id
  Description: VPC Identifier

These parameter definitions will display drop-down menus from which you can select a subnet and VPC.   CloudFormation treats your selections as strings. Our Lambda function will look up information about these values.

IAM role definition

As mentioned earlier, the IAM role for Lambda execution has to provide the necessary privileges.  In our case, this includes looking up information on VPCs and subnets as well as sending information to CloudWatch Logs.  Here are parts of the role definition that accomplish this.

Statement:
- Effect: Allow
  Action:
  - logs:CreateLogGroup
  - logs:CreateLogStream
  - logs:PutLogEvents
  Resource: arn:aws:logs:*:*:*
- Effect: Allow
  Action:
  - ec2:DescribeSubnets
  - ec2:DescribeVpcs
  Resource: "*"

Lambda function definition

I am going to focus on the parts of the Lambda function that are more specific to CloudFormation. You can read about how to write Lambda functions in the Developers Guide.

  1. CloudFormation calls the Lambda function to provision, update, or delete a resource. The function must determine why it was called and respond accordingly. To do this, check the event['RequestType'] value. In the case of our Lambda function, we are not creating any new AWS resources. We are just looking up information. This means that we don’t need to do anything extra when deleting the resource. We can just exit the function. Here are the lines of code that do this: 
    if event['RequestType'] == 'Delete':
    response_status = cfnresponse.SUCCESS
    cfnresponse.send(event, context, response_status, response_data)
  2. From the previous step, we also see that the function sets a return status, in this case cfnresponse.SUCCESS. In the case of a failure, it would return cfnresponse.FAILED. Additionally, all execution paths should end by calling cfnresponse.send(). This tells CloudFormation that the Lambda function is complete and has the given status.
  3. The function looks at the resource property called NameFilter as shown in the following line of code. We will see this later in the template when the custom resources are defined. This property will contain a stack parameter (e.g., subnet-12345 or vpc-67890). 
    name_filter = event['ResourceProperties']['NameFilter']
  4. The AWS Lambda function calls either ec2.describe_subnets or ec2.describe_vpcs to look up the resource based on the value of name_filter. The function then stores the attributes of the resource as elements of the response_data array for use by the CloudFormation template. For example, here is the code that fetches the information for a subnet, stores the values that will be returned to the CloudFormation template, and sends the information to CloudWatch Logs. 
    CidrBlock = subnets['Subnets'][0]['CidrBlock']
VpcId = subnets['Subnets'][0]['VpcId']
AvailabilityZone = subnets['Subnets'][0]['AvailabilityZone']
response_data['AvailabilityZone'] = AvailabilityZone
response_data['CidrBlock'] = CidrBlock
response_data['VpcId'] = VpcId

logger.info('subnet AvailabilityZone {}'.format(AvailabilityZone))
logger.info('subnet CidrBlock {}'.format(CidrBlock))
logger.info('subnet VpcId {}'.format(VpcId))

Resource definition

We now turn our attention to the resource section of the template. Let’s take a look at the definition of the SubnetInfo resource.

SubnetInfo:
  Type: Custom::SubnetInfo
  Properties:
    ServiceToken: !GetAtt GetAttFromParam.Arn
    NameFilter: !Ref SubnetName

 

This defines a resource named SubnetInfo. It’s a custom resource that will be provisioned by the Lambda function GetAttFromParam. Additionally, the property NameFilter will be set to the value of the SubnetName stack parameter. From our discussion of the Lambda function earlier, you can see that the NameFilter property was used in the Lambda function to identify the subnet to use in the ec2.describe_subnets API call.

Template outputs

The template output section displays the attribute values of our custom resource.  Let’s take a look at the following lines of code from that section.

SubnetAvailabilityZone:
  Description: Subnet AvailabilityZone
  Value: !GetAtt SubnetInfo.AvailabilityZone

SubnetCidrBlock:
  Description: Subnet CidrBlock
  Value: !GetAtt SubnetInfo.CidrBlock

SubnetVpcId:
  Description: Subnet VpcId
  Value: !GetAtt SubnetInfo.VpcId

In this code fragment, we call the GetAtt intrinsic on our SubnetInfo resource. The AvailabilityZone, CidrBlock, and VpcId attribute values correspond the same elements in the response_data array from our Lambda function.  These values were obtained from the ec2.describe_subnets() API operation.

Demonstration

Now let’s see how this works in practice. You can follow along by downloading the template and running it through CloudFormation. Make sure you are using an AWS Region with at least one VPC and subnet. You will incur charges for the Lambda function calls associated with the creation and deletion of the stack. These functions take very little time to run so the costs will be minimal and might fall into the free tier. I am assuming you already familiar with CloudFormation so I am just providing the main steps.

  1. Download the template from the AWS Samples GitHub repository.
  2. Start the AWS CloudFormation console. Choose the Choose File button and select your file.
  3. You are prompted for a subnet and VPC as shown in the following screenshot.
    Since the template creates an IAM role, you will need to acknowledge the prompt about the creation of IAM resources. CloudFormation takes about one minute to run the template.
  4.  After you run the template, go to the outputs tab, and you will see the results of the template.You can see from these results that the custom resources looked up the VPC ID, Availability Zone, and CIDR block of the subnet I entered, as well as the CIDR block of the VPC.

Conclusion
You can use CloudFormation custom resources to look up additional information on CloudFormation stack parameters. This technique enables you to retrieve the attributes for AWS-specific parameters such as subnets and VPCs. You can, for example, look up the Availability Zone for a subnet and use it to provision additional CloudFormation resources in that same Availability Zone. In short, CloudFormation custom resources can be the catalyst for more advanced provisioning activities.

If you have feedback about this blog post, submit comments in the Comments section that follows. If you have questions about this blog post, start a new thread on the Amazon CloudFormation forum or contact AWS Support.

About the Author

Jeff Levine is an Enterprise Solutions Architect at Amazon Web Services. He enjoys helping customers to adopt AWS for their business critical workloads.