Using Terraform to Manage AWS Programmable Infrastructures

By Alex Heneveld, CTO at Cloudsoft Corporation
By Kenneth Walsh, Sr. Partner Solutions Architect at AWS

Programmable infrastructures allow you to manage on-premises and cloud resources through code instead of with the management platforms and manual methods traditionally used by IT teams.

An infrastructure captured in code is simpler to manage, can be replicated or altered with greater accuracy, and benefits from all sorts of automation. It can also have changes to it implemented and tracked with the version control methods customarily used in software development.

Programmable infrastructures become particularly interesting when you provision workloads on a hybrid cloud environment that consists of on-premises and public cloud resources.

Two services—AWS CloudFormation and Terraform by HashiCorp—allow you to express your infrastructure resources as code and manage them programmatically.

Each has its advantages, but some enterprises already have expertise in Terraform and prefer using it to manage their Amazon Web Services (AWS) resources. To accommodate that preference, CloudFormation allows you to use non-AWS resources to manage AWS infrastructure.

In this post, we show you how to use Terraform to control your AWS resources programmatically. More specifically, we’ll walk you through the steps to create a CloudFormation registry resource type for Terraform and deploy it as an AWS Service Catalog product.

As a bonus, we’ll conclude with some recommendations for security best practices.

How it Works

AWS CloudFormation codifies the details of an infrastructure into a configuration file, referred to as a template. CloudFormation currently supports a large number of resources.

If your resource is not currently on the AWS list, CloudFormation lets you create a resource using the CloudFormation Registry.

Terraform is not on the list of currently supported resources, so Cloudsoft had to create a registry resource for it. We named it Cloudsoft::Terraform::Infrastructure. To communicate with the Terraform server, our resource uses the Secure Shell (SSH) networking protocol.

Cloudsoft is an AWS Partner Network (APN) Advanced Consulting Partner with the AWS DevOps Competency. Cloudsoft helps businesses throughout their cloud journey by providing innovative combinations of services, software, and expertise.

To set up the registry resource, you need to gather the following information beforehand:

Terraform DNS hostname or IP address
SSH KeyPair
SSH username
SSH client private key
SSH port
SSH serve public key fingerprint

Our registry resource creates and uses the following AWS Systems Manager parameters:

/cfn/terraform/logs-s3-bucket-name
/cfn/terraform/process-manager
/cfn/terraform/ssh-username
/cfn/terraform/ssh-fingerprint
/cfn/terraform/ssh-host
/cfn/terraform/ssh-key
/cfn/terraform/ssh-port

The AWS CloudFormation template acts as a proxy to Terraform. To communicate with the Terraform server, it uses a CloudFormationRegistry Cloudsoft::Terraform::Infrastructure resource type. The resulting architecture is shown in the following diagram.

Figure 1 – Architecture of Terraform customer resource on AWS CloudFormation.

Once the solution is deployed, the CloudFormation and Terraform files are placed in an Amazon Simple Storage Service (Amazon S3) bucket.

You can then launch the CloudFormation wrapper files, and also use them to create AWS Service Catalog products so end users with the proper permissions can launch them from the Service Catalog console based on the Terraform CloudFormation wrapper file.

Either way, CloudFormation uses the Cloudsoft::Terraform::Infrastructure resource to communicate with the Terraform server. After that, the Terraform server manages the AWS resources, and the resource provider logs the activity into an S3 bucket.

Add Terraform as a Custom Resource to AWS CloudFormation

The following procedures add Terraform as a registry resource to AWS CloudFormation and create an AWS Service Catalog product for others to use:

Make sure Terraform server is available.
Create AWS Identity and Access Management (AIM) roles.
Install the Cloudsoft Terraform resource.

Step 1: Make sure the Terraform server is available

This solution requires a Terraform server be deployed and available. You can use an existing Terraform server, or deploy a new Terraform server using the CloudFormation templates. The templates create a new Amazon Elastic Compute Cloud (Amazon EC2) instance and installs Terraform.

One CloudFormation resource type is required for each AWS region.

Step 2: Create IAM roles

For the Terraform server to operate, create AWS Identity and Access Management (IAM) roles in each account.

Register a CloudFormationRegistry Cloudsoft::Terraform::Infrastructure resource type in each region where you’ll use Terraform, and create these two IAM roles:

Execution role: This is the role the Terraform server will use to deploy AWS resources.
Logging role: This is the role the Terraform process will use to create log entries.

You can configure multiple accounts to use a single Terraform instance. In a Control Tower managed environment, for example, the Terraform instance can be placed in a shared services account. Other accounts can then configure their resource types to use the Terraform instance in the shared service account.

Step 3: Install the Cloudsoft Terraform resource

Instructions are in the GitHub reference architecture.

Update the Terraform Binary Version

If you become aware of a security vulnerability affecting the Terraform binary version installed on your Terraform server, switch to a version that is not affected by the vulnerability.

To do this, update your Terraform Wrapper Server stack by setting the stack’s TerraformVersion parameter to the unaffected version. Keep in mind you may need to update your Terraform configurations if a backwards incompatible change has been introduced in the Terraform binary version to which you are switching.

Watch the Video (7:15)

In this video, you’ll see how to provision resources on AWS using a CloudFormation resource type provider for Terraform. With this solution, you can pass product parameters to Terraform, provision Terraform resources on AWS, and manage resources as a group using AWS Service Catalog.

Conclusion

By creating a custom AWS CloudFormation resource for Terraform, you can control your on-premises and public cloud resources programmatically.

You can access that resource directly through the CloudFormation console, or through the AWS Service Catalog, which gives you an extra layer of governance and control.

Get more information about our solution here:

.

.

Cloudsoft – APN Partner Spotlight

Cloudsoft is an AWS DevOps Competency Partner that helps businesses throughout their cloud journey by providing innovative combinations of services, software, and expertise.

Contact Cloudsoft | Practice Overview | AWS Marketplace

*Already worked with Cloudsoft? Rate this Partner

*To review an APN Partner, you must be an AWS customer that has worked with them directly on a project.

AWS Partner Network (APN) Blog