Best practices for deploying EC2 instances with AWS CloudFormation
Deploying an Amazon Elastic Compute Cloud (Amazon EC2) instance with AWS CloudFormation is a relatively simple process. The challenges arise as you begin to customize the instance to suit your environment. What you deploy in a private subnet will have different security constraints compared with what you deploy in a public subnet.
In this post, I want to give you some guidance to ensure that no matter how you deploy an instance, you apply what I consider to be the best practices for bootstrapping an EC2 instance.
When building an instance for your infrastructure, you must consider several aspects of security and access control, and configuration.
Security and access control
To illustrate today’s topic, I have chosen one of our most used Quick Starts: The Linux bastion hosts Quick Start.
Because a bastion host is on the public side of the DMZ and is exposed to attack, it is a “hardened” server. As such, it’s a good example for highlighting instance-deployment best practices.
When I build security around an instance, I like to approach it in layers. I work my way in from the perimeter of the network to the server. Then I work in from the server to the various resources that the server will access.
During this process, I consider the following:
- Security groups
- AWS Identity and Access Management (IAM) on EC2 instances, including the following:
- Amazon CloudWatch
- Amazon Simple Storage Service (Amazon S3)
- Instance profiles
Security groups help to ensure that only those with the proper access can traverse the bastion host into your network. Use security groups inside your network to restrict access between resources.
Because the bastion host is essentially a gateway into your network, you must ensure that the appropriate port is open and that you restrict access to a specific IP address or IP range.
The port restriction will depend on what kind of instance you are deploying. It’s up to you, as the orchestrator, to determine which port or ports must be allowed input into your instance.
Because this is a bastion host and you will access the server by using Secure Shell (SSH), allow input only on port 22.
Important: Inbound rules should never specify
0.0.0.0/0. This value should be set as needed. This example relies on a parameter that is provided at runtime (
!Ref RemoteAccessCIDR) and the only allowed port is port 22.
The following diagram shows the external user accessing the bastion host.
- Specify inbound access only for the ports necessary.
- Grant inbound access only to the IP address or IP range needed. In this case, I allow the IP range 10.0.0.1/28.
You have established inbound controls for your instance by using security groups. Now I’ll show you how to use IAM to further secure your EC2 instance.
IAM on EC2 instances
Now that your instance has been secured from the outside, let’s look at the access your host needs to other resources within your environment. For a bastion host, typically the access is to other EC2 instances that have no inbound access from outside the network. Your bastion host must present valid credentials to the other instances before they in turn will grant it access. To achieve this, you can use AWS Security Token Service (AWS STS). The AWS STS service issues temporary credentials to your bastion host.
The IAM role grants the bastion host the needed AWS STS credentials to access other EC2 instances. You can also use AWS STS to request access to different AWS services.
To monitor your instances and log various events and activities, you can use CloudWatch.
As you bootstrap your bastion host, you must first create a log group. The log group is a container for the logging activity that is originated from the instance.
Now that you have created the log group, you must grant your bastion host access to act upon that log group.
Under Action, you specify the different the activities in CloudWatch that you want manage.
Under Resource, you restrict these activities to a specific log group.
Under Effect, you define whether to allow the activities to be performed.
When bootstrapping instances, you often need to access to Amazon S3 to retrieve additional software or to store the results of tasks performed on the instance. It is best to make sure that access is to S3 is scoped as tightly as possible. For the bastion host, you will need access to retrieve data from S3.
Under Action, specify the activity in Amazon S3 that you want to manage.
Under Resource, restrict the activity to a specific S3 bucket.
Under Effect, define whether to allow the activity to be performed.
To conclude the IAM section, associate your roles and policies with your instance by using the instance profile.
BastionHostProfile relies on the
BastionHostPolicy and either the
BastionHostRole that is created as part of the CloudFormation stack or a role provided at runtime (
When an EC2 instance is launched, the user data prepares the instance for use by executing scripts on the instance at startup. This is where you execute the tasks that you require for your instance build.
For a bastion host, the user data section is extensive, comprising calls to install different packages and perform updates. This is where the helper scripts come in. The helper scripts are Python scripts that assist in the buildup of your instance. These helper scripts are:
For this post, I want to focus on two of the helper scripts: cfn-init and cfn-signal.
The cfn-init script directs your build to execute the configuration as defined in the init section of your CloudFormation template.
The cfn-init script invokes the AWS::CloudFormation::Init type. Typically, the CloudFormation Init contains multiple config sets, and the cfn-init script will invoke these in the desired order of execution. However, when there is only a single config set inside the CloudFormation Init, there is no need to specify the config set.
For this application, the command:
is the same as
The cfn-init directs the execution of the config sets inside of the CloudFormation Init.
The single config set (
config) in this example includes two sections:
files section retrieves a file from Amazon S3. The
commands section executes the file and passes along relevant parameters based on previous inputs.
Now that the tasks are executing, you must notify AWS CloudFormation when the tasks are complete and whether they are successful. For this, you can use cfn-signal.
The signaling process in a CloudFormation template is composed of two parts:
- A creation policy that tells the CloudFormation template how many signals to expect
- The signal itself
When the CloudFormation template execution comes across a creation policy, it halts execution of its tasks until the number of signals specified in the creation policy is received or a timeout occurs.
If the signal is a failure or a timeout occurs, the stack launch fails and, if appropriate, the stack is rolled back.
In this example, the creation policy indicates to AWS CloudFormation to wait for a signal from each of the bastion hosts being deployed, or to time out after 30 minutes.
- 'cfn-signal -e $? --stack ' - !Ref 'AWS::StackName' - ' --resource BastionAutoScalingGroup --region ' - !Ref 'AWS::Region'
- The creation policy in the Auto Scaling group notifies AWS CloudFormation of the number of signals and timeouts to expect.
- When an instance comes online, a signal is sent to AWS CloudFormation.
- AWS CloudFormation halts execution of subsequent tasks until the number of signals expected is received or a timeout occurs.
In this overview of best practices for deploying EC2 instances by using AWS CloudFormation, I covered the following:
- Protecting your outer perimeter with security groups
- Controlling access to other account resources by using IAM
- Using various aspects of user data and cfn-init scripts to perform bootstrapping operations on your instances
- Enabling signaling within your stack to let AWS CloudFormation know if the bootstrapping operations completed successfully
The key takeaway from this post is to always consider what resources and services your instance needs to access, and to scope your permissions tightly around those requirements.
I hope that seeing these bootstrapping best practices in context within an existing Quick Start will help you understand how the pieces come together to form a production-level deployment.