Speed up instance bootstrapping by using dynamically created images

When building automated deployments of Amazon EC2 instances, you have a couple of options. One is to use an Amazon Machine Instance (AMI) that contains all your pre-baked dependencies. A second is to use a base image and download software from a public source at the time of instance launch. Downloading public binaries during a CloudFormation launch of an EC2 instance, however, can be slow and result in delayed bootstrapping of instances. This can present challenges if such instances are part of an AWS Auto Scaling group.

Using pre-baked AMIs can add to the overhead of keeping AMIs updated with the latest software releases. I prefer to use dynamic bootstrapping of instances wherever possible.

In this blog, I discuss a Quick Start pattern that uses the latest software binaries while providing fast bootstrapping of the automatically scaled instance.

This approach is a work-around for relatively large binary downloads where installing software dynamically creates a noticeable lag in the instance bootstrap. For the configuration, you can limit the image creation to include only the latest binaries. You can then leave the configurable options as part of the Auto Scaling group launch configuration.

If you want to jump right into the source code, take a look at the Magento [no longer available] and Drupal Quick Starts. [The Magento link points to a line of code in a repo that has been archived and is now private and read-only.]

Solution design

Here is an overview of the solution.

Create a new Amazon EC2 instance (including all the binaries downloaded at runtime) with the desired configuration using cfn-init and user-data.
Create an AWS Lambda function that can create an AMI from an Amazon EC2 instance ID, which is provided as input.
Create a CloudFormation custom resource that invokes the Lambda function to build the AMI.
Use the newly built AMI as the ImageId in the Auto Scaling group launch configuration.
Create a Lambda function to deregister the AMI at the time of stack deletion.
Create a CloudFormation custom resource that invokes the Lambda function to deregister the image.

With this approach, you perform a one-time build of the AMI when you launch your CloudFormation stack. The bootstrapping will perform quickly even as the Auto Scaling group adds new instances.

Let’s get into each of the steps, taking the Drupal Quick Start as an example. While I provide snippets inline for each step, I also link them to the original templates so you can review the fully functioning code.

1. Create a new Drupal instance

The Quick Start gives users a choice of installing PHP and Drupal versions and then downloads the selected software packages. This is done by building cfn-init config sets that correspond to a chosen version. A snippet (converted to YAML) is shown here. PhpVersion and DrupalVersion are input parameters.

configSets:
  install_drupal:
    - install_cfn
    - mount_efs
    - createdb
    - !Join
      - ''
      - - install-packageSet-
        - !Ref 'PhpVersion'
    - drupal
    - !If
      - EnableElastiCacheCondition
      - !Join
        - ''
        - - memcached-drupal-
          - !Ref 'DrupalVersion'
      - !Ref 'AWS::NoValue'
  drupal_cdn_module:
    - download-cdn-module
    - !Join
      - ''
      - - install-cdn-module-
        - !Ref 'DrupalVersion'

2. Create a Lambda function to create an AMI

Create a Lambda function by using Python 2.7 runtime that uses the Boto3 SDK to create an image from a provided instance ID. Here is the snippet of code that creates the image.


	ec2 = boto3.resource('ec2')
    instance = ec2.Instance(event['ResourceProperties']['InstanceId'])
    instance.stop()
    instance.wait_until_stopped()
    image = instance.create_image(Name="%s-%i" % (event['ResourceProperties']['AMIName'], randint(0,100000)), Description="Created with Cloudformation")
    boto3.client('ec2').get_waiter('image_available').wait(ImageIds=[image.id])
    responseData = {'ami':image.id}
    cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData)

3. Create a CloudFormation custom resource to invoke image creation

Next, create a CloudFormation custom resource to invoke the CreateAMI Lambda function. The definition of the custom resource is as follows.


CreateAMI:
  Type: Custom::CreateAMI
  DeletionPolicy: Retain
  Properties:
    ServiceToken: !GetAtt 'CreateAMIFunction.Arn'
    InstanceId: !Ref 'EC2InstanceId'
    AMIName: !Ref 'AMIBaseName'
    Version: 1

4. Use the new image in the Auto Scaling group

In the Auto Scaling group launch configuration, use the AMI ID of the built image.


WebServerLC:
  Type: AWS::AutoScaling::LaunchConfiguration
  DependsOn: CreateDrupalAMI
  Properties:
    ImageId: !GetAtt 'CreateDrupalAMI.Outputs.AMIId'

5. Create a Lambda function to deregister an AMI

Now create a Lambda function to deregister the image. This deletes unneeded resources and allows a clean deletion of the CloudFormation stack. Again, use the Boto3 SDK for Python to call the deregister API. The main snippet of code follows.

  if event['RequestType'] == 'Delete':
    try:
      ec2 = boto3.resource('ec2')
      image = ec2.Image(event['ResourceProperties']['AMIId'])
      image.deregister()
      cfnresponse.send(event, context, cfnresponse.SUCCESS, {})
    except Exception as inst:
      print(inst)
      cfnresponse.send(event, context, cfnresponse.FAILED, {})
  else:
    cfnresponse.send(event, context, cfnresponse.SUCCESS, {})

Notice that the deregister operation occurs only when the CloudFormation stack deletion event is observed, that is, [‘RequestType’] == ‘Delete’.

6. Create a CloudFormation custom resource to invoke image deregistration

Finally, create a new CloudFormation custom resource to invoke the Lambda function that deregisters the image. The definition of the custom resource is as follows.


CleanUpAMI:
  Type: Custom::CleanUpAMI
  Properties:
    ServiceToken: !GetAtt 'CleanUpAMIFunction.Arn'
    AMIId: !GetAtt 'CreateAMI.ami'
    Version: 1

Conclusion

I provided a simple method of building an AMI from an Amazon EC2 instance at CloudFormation launch time to download the latest binaries. Then I used the AMI as the ImageId in the Auto Scaling group launch configuration for fast bootstrapping of the instances created by the Auto Scaling group.

The Quick Start team has used this pattern in both the Magento [no longer available] and Drupal Quick Starts. [The Magento link points to a line of code in a repo that has been archived and is now private and read-only.] For more best practices when building automated deployments, please review our contributor’s guide. If you have any feedback about the steps in this post, please let us know in the comments.

Integration & Automation