AWS CloudFormation Update – YAML, Cross-Stack References, Simplified Substitution

AWS CloudFormation gives you the ability to express entire stacks (collections of related AWS resources) declaratively, by constructing templates. You can define a stack, specify and configure the desired resources and their relationship to each other, and then launch as many copies of the stack as desired. CloudFormation will create and set up the resources for you, while also taking care to address any ordering dependencies between the resources.

Today we are making three important additions to CloudFormation:

• YAML Support – You can now write your CloudFormation templates in YAML.
• Cross Stack References – You can now export values from one stack and use them in another.
• Simplified Substitution – You can more easily perform string replacements within templates.

Let’s take a look!

YAML Support
You can now write your CloudFormation templates in YAML (short for YAML Ain’t Markup Language). Up until now, templates were written in JSON. While YAML and JSON have similar expressive powers, YAML was designed to be human-readable while JSON was (let’s be honest) not. YAML-based templates use less punctuation and should be substantially easier to write and to read. They also allow the use of comments. CloudFormation supports essentially all of YAML, with the exception of hash merges, aliases, and some tags (binary, imap, pairs, TIMESTAMP, and set).

When you write a CloudFormation template in YAML, you will use the same top-level structure (Description, Metadata, Mappings, Outputs, Parameters, Conditions, and Resources).  Here’s what a parameter definition looks like:

Parameters:
  DBName:
    AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
    ConstraintDescription: must begin with a letter and contain only alphanumeric
      characters.
    Default: wordpressdb
    Description: The WordPress database name
    MaxLength: '64'
    MinLength: '1'
    Type: String

When you use YAML, you can also use a new, abbreviated syntax to refer to CloudFormation functions such as GetAtt, Base64, and FindInMap. You can now use the existing syntax ("Fn::GetAtt") or the new, tag-based syntax (!GetAtt). Note that the “!” is part of the YAML syntax for tags; it is not the “logical not” operator. Here’s the old syntax:

- Fn::FindInMap:
    - AWSInstanceType2Arch
    - Ref: InstanceType
    - Arch

And the new one:

!FindInMap [AWSInstanceType2Arch, !Ref InstanceType, Arch]

As you can see, the newer syntax is shorter and cleaner. Note, however, that you cannot put two tags next to each other. You can intermix the two forms and you can also nest them. For example, !Base64 !Sub is invalid but !Base64 Fn::Sub is fine.

The CloudFormations API functions (CreateChangeSet, CreateStack, UpdateStack, and so forth) now accept templates in either JSON or YAML. The GetTemplate function returns the template in the original format. The CloudFormation designer does not support YAML templates today, but this is on our roadmap.

Cross Stack References
Many AWS customers use one “system” CloudFormation stack to set up their environment (VPCs, VPC subnets, security groups, IP addresses, and so forth) and several other “application” stacks to populate it (EC2 & RDS instances, message queues, and the like). Until now there was no easy way for the application stacks to reference resources created by the system stack.

You can now create and export values from one stack and make use of them in other stacks without going to the trouble of creating custom CloudFormation resources. The first stack exports values like this:

Outputs: 
  TSSG: 
    Value: !Ref TroubleShootingSG
    Export:
      Name: AccountSG

The other stacks then reference them using the new ImportValue function:

EC2Instance:
  Type: AWS::EC2::Instance
  Properties:
    SecurityGroups:
      - !ImportValue AccountSG

The exported names must be unique with the AWS account and the region. A stack that is referenced by another stack cannot be deleted and it cannot modify or remove the exported value.

Simplified Substitution
Many CloudFormation templates perform some intricate string manipulation in order to construct command lines, file paths, and other values that cannot be fully determined until the stack is created.  Until now, this required the use of fn::Join. In combination with the JSON syntax, this results in some messy templates that were hard to understand and maintain. In order to simplify this important aspect of template development, we are introducing a new substitution function, fn::Sub. This function replaces variables (denoted by the syntax ${variable_name}) with their evaluated values. For example:

configure_wordpress:
  commands:
    01_set_mysql_root_password:
      command: !Sub |
           mysqladmin -u root password '${DBRootPassword}'
      test: !Sub |
           $(mysql ${DBName} -u root --password='${DBRootPassword}' >/dev/null 2>&1 </dev/null); (( $? != 0 ))
    02_create_database:
      command: !Sub |  
           mysql -u root --password='${DBRootPassword}' < /tmp/setup.mysql
      test: !Sub |
           $(mysql ${DBName} -u root --password='${DBRootPassword}' >/dev/null 2>&1 </dev/null); (( $? !=0))

If you need to generate ${} or ${variable}, simply write ${!} or ${!variable}.

Coverage Updates
As part of this release we also added additional support for AWS Key Management Service (AWS KMS), EC2 Spot Fleet, and Amazon EC2 Container Service. See the CloudFormation Release History for more information.

Available Now
All of these features are available now and you can start using them today!

If you are interested in learning more about CloudFormation, please plan to attend our upcoming webinar, AWS Infrastructure as Code. You will learn how to take advantage of best practices for planning and provisioning your infrastructure, and you will have the opportunity to see the new features in action.

— Jeff;