Using Terraform to Manage AWS Programmable Infrastructures
By Alex Heneveld, CTO at Cloudsoft Corporation
By Kenneth Walsh, Sr. Partner Solutions Architect at AWS
Programmable infrastructures allow you to manage on-premises and cloud resources through code instead of with the management platforms and manual methods traditionally used by IT teams.
An infrastructure captured in code is simpler to manage, can be replicated or altered with greater accuracy, and benefits from all sorts of automation. It can also have changes to it implemented and tracked with the version control methods customarily used in software development.
Programmable infrastructures become particularly interesting when you provision workloads on a hybrid cloud environment that consists of on-premises and public cloud resources.
Each has its advantages, but some enterprises already have expertise in Terraform and prefer using it to manage their Amazon Web Services (AWS) resources. To accommodate that preference, CloudFormation allows you to use non-AWS resources to manage AWS infrastructure.
In this post, we show you how to use Terraform to control your AWS resources programmatically. More specifically, we’ll walk you through the steps to create a CloudFormation registry resource type for Terraform and deploy it as an AWS Service Catalog product.
As a bonus, we’ll conclude with some recommendations for security best practices.
How it Works
AWS CloudFormation codifies the details of an infrastructure into a configuration file, referred to as a template. CloudFormation currently supports a large number of resources.
If your resource is not currently on the AWS list, CloudFormation lets you create a resource using the CloudFormation Registry.
Terraform is not on the list of currently supported resources, so Cloudsoft had to create a registry resource for it. We named it
Cloudsoft::Terraform::Infrastructure. To communicate with the Terraform server, our resource uses the Secure Shell (SSH) networking protocol.
Cloudsoft is an AWS Partner Network (APN) Advanced Consulting Partner with the AWS DevOps Competency. Cloudsoft helps businesses throughout their cloud journey by providing innovative combinations of services, software, and expertise.
To set up the registry resource, you need to gather the following information beforehand:
- Terraform DNS hostname or IP address
- SSH KeyPair
- SSH username
- SSH client private key
- SSH port
- SSH serve public key fingerprint
Our registry resource creates and uses the following AWS Systems Manager parameters:
The AWS CloudFormation template acts as a proxy to Terraform. To communicate with the Terraform server, it uses a
CloudFormationRegistry Cloudsoft::Terraform::Infrastructure resource type. The resulting architecture is shown in the following diagram.
Figure 1 – Architecture of Terraform customer resource on AWS CloudFormation.
Once the solution is deployed, the CloudFormation and Terraform files are placed in an Amazon Simple Storage Service (Amazon S3) bucket.
You can then launch the CloudFormation wrapper files, and also use them to create AWS Service Catalog products so end users with the proper permissions can launch them from the Service Catalog console based on the Terraform CloudFormation wrapper file.
Either way, CloudFormation uses the
Cloudsoft::Terraform::Infrastructure resource to communicate with the Terraform server. After that, the Terraform server manages the AWS resources, and the resource provider logs the activity into an S3 bucket.
Add Terraform as a Custom Resource to AWS CloudFormation
The following procedures add Terraform as a registry resource to AWS CloudFormation and create an AWS Service Catalog product for others to use:
- Make sure Terraform server is available.
- Create AWS Identity and Access Management (AIM) roles.
- Install the Cloudsoft Terraform resource.
Step 1: Make sure the Terraform server is available
This solution requires a Terraform server be deployed and available. You can use an existing Terraform server, or deploy a new Terraform server using the CloudFormation templates. The templates create a new Amazon Elastic Compute Cloud (Amazon EC2) instance and installs Terraform.
One CloudFormation resource type is required for each AWS region.
Step 2: Create IAM roles
For the Terraform server to operate, create AWS Identity and Access Management (IAM) roles in each account.
CloudFormationRegistry Cloudsoft::Terraform::Infrastructure resource type in each region where you’ll use Terraform, and create these two IAM roles:
- Execution role: This is the role the Terraform server will use to deploy AWS resources.
- Logging role: This is the role the Terraform process will use to create log entries.
You can configure multiple accounts to use a single Terraform instance. In a Control Tower managed environment, for example, the Terraform instance can be placed in a shared services account. Other accounts can then configure their resource types to use the Terraform instance in the shared service account.
Step 3: Install the Cloudsoft Terraform resource
Instructions are in the GitHub reference architecture.
Update the Terraform Binary Version
If you become aware of a security vulnerability affecting the Terraform binary version installed on your Terraform server, switch to a version that is not affected by the vulnerability.
To do this, update your Terraform Wrapper Server stack by setting the stack’s
TerraformVersion parameter to the unaffected version. Keep in mind you may need to update your Terraform configurations if a backwards incompatible change has been introduced in the Terraform binary version to which you are switching.
Watch the Video (7:15)
In this video, you’ll see how to provision resources on AWS using a CloudFormation resource type provider for Terraform. With this solution, you can pass product parameters to Terraform, provision Terraform resources on AWS, and manage resources as a group using AWS Service Catalog.
By creating a custom AWS CloudFormation resource for Terraform, you can control your on-premises and public cloud resources programmatically.
You can access that resource directly through the CloudFormation console, or through the AWS Service Catalog, which gives you an extra layer of governance and control.
Get more information about our solution here:
- AWS CloudFormation Connector for Terraform
- Introduction to Terraform
- CloudFormation Command Line Interface (CLI)
Cloudsoft – APN Partner Spotlight
Cloudsoft is an AWS DevOps Competency Partner that helps businesses throughout their cloud journey by providing innovative combinations of services, software, and expertise.
*Already worked with Cloudsoft? Rate this Partner
*To review an APN Partner, you must be an AWS customer that has worked with them directly on a project.