AWS CodeDeploy Deployments with HashiCorp Consul

The following is a guest post from Cameron Stokes, Solutions Engineer at AWS DevOps Competency Partner HashiCorp. 

AWS CodeDeploy automates code deployments to Amazon Elastic Compute Cloud (Amazon EC2) and on-premises instances, allowing developers to customize deployment steps as needed per their application requirements. HashiCorp Consul is an open-source tool providing service discovery and orchestration for modern applications.

Used together, CodeDeploy and Consul can enable developers to confidently deploy their applications while helping to ensure the health and stability of their overall application environment.

In this post, we’ll demonstrate how Consul can complement CodeDeploy by:

• Registering applications for service discovery during deployment
• Marking applications and nodes as “under maintenance” during deployments
• Halting deployments if application instances become unhealthy
• Providing configuration data to applications

We will demonstrate how Consul and AWS CodeDeploy can be used together to quickly deploy applications in a dynamic environment without impacting application stability.

Setup

To get started, clone the repository to your local workstation, and then follow the instructions in the aws-codedeploy-consul README. To assist with the initial setup of the application environment (including VPC, instances, and CodeDeploy resources), we’ll use Terraform for provisioning our AWS resources and Atlas for bootstrapping our Consul cluster, both provided by HashiCorp.

Once your demo environment is created, you can deploy the SampleApp_Linux_Consul application by following the instructions in the output from the terraform apply command.

Integration Details

With Consul available for assisting with deployments, we can customize our CodeDeploy application specification file (AppSpec file) for service discovery, health monitoring, and application configuration.

CodeDeploy Deployment Configuration

To set up our application and register it with Consul, we define a few files to install as a part of our deployment in the files section of our AppSpec file. The files are:

• index.html.ctmpl – The web page for our application. The extension .ctmpl is used to indicate that this is a template file that we’ll use to configure our application during installation. This is explained in more detail later in this blog post.
• health – A file used for determining the health of our application.
• sampleapp.json – The service definition file for registering our service with Consul.

These files are installed during the deployment’s Install event, and by customizing other events in the deployment lifecycle, we can integrate our application with Consul.

The event hooks and scripts that are executed are:

• ApplicationStop
  • scripts/consul_enable_maintenance – Enables “maintenance mode” on the node, which removes the node from service lookups via Consul. When the node is in maintenance mode, dependent systems in our application environment will stop sending requests to the web server on this node.
  • scripts/stop_server – Stops the web server.
• BeforeInstall
  • scripts/install_dependencies – Installs our application’s dependencies, such as our web server.
  • scripts/start_server – Starts the web server.
• AfterInstall
  • scripts/consul_reload – Instructs Consul to reload its configuration in case our service or healthcheck definitions have changed.
• ApplicationStart
  • scripts/consul_update_from_kv – Updates our application configuration from Consul’s Key/Value datastore. Consul’s Key/Value datastore is explained in more detail later in this blog post.
  • scripts/consul_disable_maintenance – Disables “maintenance mode” on the node, and re-adds the node to Consul’s service catalog.
• ValidateService
  • scripts/consul_validate_health – Verifies that Consul perceives our updated application as healthy, which allows other dependent systems to send requests to the web server on this node.

If any of these scripts fails, such as consul_validate_health determining that our application is not healthy within Consul, the CodeDeploy deployment will halt and prevent further instances from being updated. This allows you to troubleshoot the failed deployment and push a fixed application revision or redeploy a previous stable revision.

Service Discovery Integration

To register our application with Consul for service discovery, we define the name of our application, its port, and healthcheck configuration in a service definition file. As defined in our AppSpec, the service definition file (/conf/sampleapp.json) is installed, and the Consul agent reloads its configuration in the AfterInstall event hook. Once the configuration is reloaded, Consul will begin monitoring our application and make it it available for service discovery.

More details on the service definition file format are in Consul’s documentation.

Application Configuration

In addition to providing service discovery functionality, Consul also provides a Key/Value store. This can be used to hold dynamic configuration and supports distributed systems such as leader election and check-and-set operations.

We can use Consul’s Key/Value store to configure our application during deployment. During our initial setup with Terraform, the Key/Value store was populated with the following data:

By prefixing our keys with our CodeDeploy deployment group name SampleApp_Linux_Consul, we can configure multiple applications or environments using the same Consul cluster.

To configure our application, we use the tool consul-template. This tool provides a convenient way to populate values from Consul to the file system using a file template.

During the ApplicationStart event hook, the script consul_update_from_kv calls consul-template specifying /tmp/index.html.ctmpl as our template and writing the rendered template to the document root of our web server /var/www/html/index.html

Our template uses the DEPLOYMENT_GROUP_NAME environment variable provided by the CodeDeploy agent to determine the Key/Value path prefix to use for configuring our application. Again, this allows you to configure multiple applications or environments with CodeDeploy and Consul.

The template then iterates over the range of keys and outputs a table of our application configuration data.

You can edit the Key/Value data with the Consul Web UI URL linked in the terraform apply output, and on your next application deployment you will see the updated configuration data. consul-template can also be configured to run as a service, providing faster and dynamic configuration updates to your application.

Conclusion

In this post we’ve demonstrated how AWS CodeDeploy and HashiCorp Consul can be a great combination for engineers to confidently and reliably deploy modern service-based applications.

Note: Be sure to follow the teardown instructions in the project README to delete the resources we’ve created so that you will not be charged for them going forward. Also be sure to remove any remaining sample application revisions in your Amazon S3 bucket that are no longer needed.

The content and opinions in this blog are those of the third party author and AWS is not responsible for the content or accuracy of this post.