AWS DevOps Blog

Building AWS CloudFormation Templates Using CloudFormer

In this week’s post Chris Whitaker, AWS Senior Manager of Software Development, will discuss best practices for building CloudFormation templates with the CloudFormer tool.

AWS CloudFormation enables you to create and manage AWS infrastructure deployments in a predictable and repeatable way using templates. Once you have a template, you can use it to deploy any number of stacks in the same region (e.g., to deploy identical configurations for test, development, and production) as well as in several regions (e.g., to serve your customers in the US and in Europe). While you can create templates from scratch or use the built-in template editors provided by the AWS toolkits for Microsoft Visual Studio and Eclipse, you may already have a running application that you want to deploy repeatedly and reliably. The CloudFormer tool helps you to build a template from a running version of your application.

To use CloudFormer, launch a CloudFormation stack. For more details, the AWS CloudFormation User Guide walks you through starting up and using CloudFormer to generate a template.

Once you have CloudFormer running, you simply select existing resources that you want to include in the template and CloudFormer will generate a template from the configuration. It tries to anticipate what you need by automatically selecting any related resources. For example, if you select an elastic load balancer, CloudFormer automatically selects the Amazon Elastic Compute Cloud (EC2) instances, auto scaling groups, EC2 security groups and so on that are connected to the load balancer.

You can choose to deselect any of the dependent resources if you don’t want to include them in the template, in which case CloudFormer will insert a link to the real resource. This is useful, for example, if you have a shared Amazon EC2 security group that you want to associate with your EC2 instances in your new stack but don’t want to have a copy of the security group for each stack created from your template. When you finish selecting your resources, you will have a functional AWS CloudFormation template that you can use to create a new stack.

Customizing CloudFormer-built Templates

Templates created by CloudFormer contain a full specification of the selected resources. For example, if you select an auto scaling group, the template will include all of the properties associated with the running auto scaling group, including such things as the specific Amazon EC2 Availability Zones configured for the auto scaling group. This may be important if you are using an EC2 Reserved Instance. However, you more likely want to generalize the template so it can be used to create stacks in other regions. In other cases, you may want to have the template user specify the value of a property when the stack is created using template parameters, such as the EC2 key pair name.

The following example shows the before and after template snippets for changing the generated template to include a call to the AWS CloudFormation–provided Fn::GetAZs intrinsic function to return the list of Availability Zones in the current region.

Here is a template snippet from CloudFormer capturing an auto scaling group:

:
"Resources" : {
  "WebServerGroup" : {
    "Type" : "AWS::AutoScaling::AutoScalingGroup",
    "Properties" : {
      :
      "AvailabilityZones" : ["us-east-1a", "us-east-1b", "us-east-1c"],                
      "LaunchConfigurationName" : { "Ref" : "LaunchConfig" },
      "MinSize" : "2",
      "MaxSize" : "2",
      :
    }
  },
 
Here are the changes to the auto scaling group to generalize the Availability Zones using intrinsic functions. In this case we use the Fn::GetAZs function to return the set of Amazon EC2 Availability Zones for the template’s intended region:
 
:
"Resources" : {
  "WebServerGroup" : {
    "Type" : "AWS::AutoScaling::AutoScalingGroup",
    "Properties" : {
      :
      "AvailabilityZones" : { "Fn::GetAZs" : "" },               
      "LaunchConfigurationName" : { "Ref" : "LaunchConfig" },
      "MinSize" : "2",
      "MaxSize" : "2",
      :
    }
  },

In some case, you will want to allow users to customize the stack when it is created by entering values such as the name of the Amazon EC2 key pair needed to log into the EC2 instances in a given stack. You may not want this to be hard coded in the template, so the following template snippets show how to add a parameter to the generated template and flow the value to the EC2 instance properties:

Here is a template snippet for an Amazon EC2 instance from CloudFormer:

:
"Resources": {
  "instance1": {
    "Type": "AWS::EC2::Instance",
    "Properties": {
      "AvailabilityZone": "us-east-1d",
      "DisableApiTermination": "FALSE",
      "ImageId": "ami-12345678",
      "InstanceType": "m1.small",
      "KernelId": "aki-87654321",
      "KeyName": "MyKeyPair",
      "Monitoring": "false",
      "SecurityGroups": [{"Ref": "sgfoo"}]      
    }
  },
:
 
Changes to the template let the user enter the Amazon EC2 key pair name as a parameter. The changes also remove the AvailabilityZone property from the instance to allow for an untargeted EC2 instance launch. (Unless you have a reason to launch in a specific EC2 availability zone, we recommend using untargeted launches.)
 
:
"Parameters" : {
  "KeyName" : {
    "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance",       
    "Type" : "String"
  }
},
:
"Resources": {
  "instance1": {
    "Type": "AWS::EC2::Instance",
    "Properties": {
      "DisableApiTermination": "FALSE",
      "ImageId": "ami-12345678",
      "InstanceType": "m1.small",
      "KernelId": "aki-87654321",
      "KeyName": {"Ref": "KeyName"},
      "Monitoring": "false",
      "SecurityGroups": [{"Ref": "sgfoo"}]      
    }
  },
:
 
Although CloudFormer can access the AWS resources in your account, it does not analyze the software deployed on the Amazon EC2 instances. If you want the template to include the software configuration on the instances using template metadata and the AWS CloudFormation on-host helper scripts, you need to add those manually. To configure the software on the instances, you can add the appropriate metadata sections as well as add user data scripts to either your EC2 instances or your auto scaling launch configurations.
 
The following snippet shows how to add metadata to capture the on-host configuration for your instances and the changes needed to call the AWS CloudFormation on-host helper scripts at deployment time. The example installs the Apache web server on the instance and adds a WaitCondition to the template to ensure the stack is not considered complete until the web server is installed.
 
:
"Resources": {
  "instance1": {
    "Type": "AWS::EC2::Instance",
    "Metadata" : {
      "AWS::CloudFormation::Init" : {
        "config" : {
          "packages" : {
            "yum" : {
              "httpd"     : []
            }
          },
          "services" : {
            "sysvinit" : {
              "httpd" : {"enabled" : "true", "ensureRunning" : "true"}                      
            }
          }
        }       
      }
    },
    "Properties": {
      "DisableApiTermination": "FALSE",
      "ImageId": "ami-12345678",
      "InstanceType": "m1.small",
      "KernelId": "aki-87654321",
      "KeyName": "MyKeyPair",
      "Monitoring": "false",
      "SecurityGroups": [{"Ref": "sgfoo"}],
      "UserData” : { "Fn::Base64" : { "Fn::Join" : ["", [            
        "#!/bin/bashn",
        "yum update -y aws-cfn-bootstrapn",
        "/opt/aws/bin/cfn-init --stack ", { "Ref" : "AWS::StackId" },
        "                      --resource instance1 ",
        "                      --region ", { "Ref" : "AWS::Region" }, "n",                   
        "/opt/aws/bin/cfn-signal -e $? '", { "Ref" : "WaitHandle" }, "'n"            
      ]]}}
    }
  },
  "WaitHandle" : {
    "Type" : "AWS::CloudFormation::WaitConditionHandle"
   },
   "WaitCondition" : {
     "Type" : "AWS::CloudFormation::WaitCondition",
     "DependsOn" : "instance1",
     "Properties" : {
       "Handle" : {"Ref" : "WaitHandle"},
       "Timeout" : "300"
    }
  },
:
 
For more about using template metadata see the AWS CloudFormation User Guide.
 

Building Best Practices with CloudFormer

CloudFormer lets you select any or all of your provisioned AWS resources. As such, you can use CloudFormer to create templates that encapsulate your common or best-practice configurations for key parts of your infrastructure. For example, you may have set up an Amazon Virtual Private Cloud (VPC) configuration just the way you want it with subnets, routing tables, network ACLs and so forth using the AWS Management Console or the VPC command-line tools. CloudFormer enables you to capture a snapshot of just those resources to build a template for recreating your configuration the next time you need it.
 
You may also have best practices around AWS resources, for example, an Amazon Relational Database Service (RDS) database configuration with DB parameter group settings and CloudWatch alarms that you want to reuse. By selecting only those resources that represent your best-practice configuration, you can either create a template that you can reuse through AWS CloudFormation substacks or create a template that represents a common starting point for each RDS-based application you want to deploy.
 
One word of caution when using CloudFormer to generate reusable templates: If you create a template that does not contain all of the dependent resources, the generated template will be dependent on the environment remaining consistent unless you either edit the template or a parameterize property values. For example, lets say you have an Amazon EC2 instance in a Amazon VPC configuration running in your environment. If you use CloudFormer to generate a template that contains just the instance and not the VPC configuration, then the generated template will contain property values tied to the specific VPC in which the instance was running. If you later delete the VPC, the generated template will no longer work because it will contain the identifiers of the subnets in the original VPC.

Summary

In conclusion, although CloudFormer produces a complete, working AWS CloudFormation template that can be used to create stacks, in most cases you should view the generated template as a starting point for generalizing and enhancing for future needs. Not only does CloudFormer let you create complete applications, but it also encapsulates and codifies your best-practice configurations for reusable pieces of a solution.