AWS Cloud Operations & Migrations Blog

Deploy Multi-Account Amazon CloudWatch Dashboards

Organizations building modern applications require a way to gain actionable insights into their Amazon Elastic Compute Cloud (Amazon EC2) workloads. Amazon CloudWatch is a monitoring and observability service that collects operational data from logs, metrics, and events. The service lets customers monitor your resources spread across different accounts or regions in a single view, visualize the operational data that it collects, and create customized views of the metrics and alarms for your AWS resources.

This post will demonstrate how you can quickly provision a multi-account CloudWatch dashboard by leveraging the Amazon CloudWatch Monitoring Framework and enforcing standardization via the AWS Organizations tag policy approach. Tag policies are a type of policy that can help you standardize tags across resources in your organization’s accounts. In a tag policy, you specify tagging rules applicable to resources when they are tagged. Creating CloudWatch dashboards and widgets can be straightforward, but creating and configuring new dashboards for every workload is time-consuming. Amazon CloudWatch Monitoring Framework automates the process of dashboard setup and provides reference config files for some of the most common workloads. The framework deploys from a your central management account with AWS Organizations and tagging enabled, and it will use AWS CloudFormation StackSets for multi-account deployment. CloudFormation StackSets extends the capability of stacks by enabling you to create, update, or delete stacks across multiple accounts and AWS Regions with a single operation.

Solution overview

In this post, you’ll automatically create CloudWatch Dashboard widgets for an Amazon EC2 instance, running a simple Apache server by tagging the Amazon EC2 instances. First, you’ll deploy the CloudWatch monitoring Framework CloudFormation template using CloudFormation StackSets to the workload account in development and staging environments. Then, you’ll deploy a stack that will create an application running on an Apache webserver.

The architecture diagram demonstrates how the solution is deployable with CloudFormation StackSets with AWS Organizations tag policies.
Figure 1: Solution architecture and workflow

You’ll complete the following actions as part of the solution:

  • Amazon EC2 instances are tagged following our AWS Organizations tag policy.
  • tagHandler AWS Lambda function fetches instances with the tag and updates the AWS Systems Manager Parameter Store.
  • Systems Manager triggers the CloudWatch Events rule, and this invokes the dashboardHandler Lambda function.
  • dashboardHandler Lambda function reads the Systems Manager parameter and updates the CloudWatch dashboard widgets

Prerequisites

The following prerequisites are required:

  1. A management account with AWS Organizations enabled. Review Tutorial: Creating and configuring an organization for detailed steps.
  2. AWS workload accounts (development and staging) for deploying the solution
    • Account ID
    • Deployment region
  3. You must create the AWS Identity and Access Management (IAM) roles required by CloudFormation StackSets to deploy across accounts and regions. These roles are necessary to establish a trusted relationship between the account that you’re administering the stack set from and the account to which you’re deploying stack instances. Refer to Grant self-managed permissions to understand the requirements for CloudFormation StackSets. There are three options to set up the permissions required to perform CloudFormation StackSet operations across multiple accounts. Review Prerequisites for stack set operations for detailed prerequisite steps for each option. For this post, we’ll be using self-managed permissions.
  4. Enable Tag Policies in AWS Organization. You can refer to Creating, updating, and deleting tag policies to see how to enable and work with tag policies.
  5. Download the CloudWatch Monitoring Framework template.
  6. Download the Apache workload template.

Solution walkthrough

The following section will walk you through the solution steps.

  1. Navigate to the AWS Cloud formation console, choose StackSets from the navigation pane, then choose Create Stackset.Demonstration of creating a StackSet from the AWS CloudFormation console.
    Figure 2: Creating the CloudFormation StackSet.
  2. You will use self-managed permissions. If using the same option, provide the StackSet Admin and Execution Role that were created as part of the prerequisite.Demonstration of configuring the self-managed permissions IAM admin ARN and execution role name. Figure 3: Setting permissions for CloudFormation StackSets.
  3. Under Prerequisite – Prepare template, choose Template is ready. Under Specify template, choose Upload Template. Choose file and provide the template that you previously downloaded, and choose Next.
  4. In the next prompt, under StackSet name, enter amazon-cloudwatch-monitoring. Fill the parameters as shown. Then choose Next.
    • Workload Name: Apache
    • Tag Schema: {"Key":"CW-Dashboard","Value":"Apache"}
    • CW Dashboard Name: ApacheDashboard
    • Access Log Group: /cw-monitoring-framework/apache/access
    • SSM Parameter Name: /cw-monitoring-framework/ApacheInstances
  5. Under Deployment locations, choose Deploy stacks in accounts, and provide the workload account ids in the box.Demonstration of choosing deployment locations and where to provide account ids. Figure 4: Configure the deployment options.
  6. Choose the region to which you would like to deploy the stack. Provide the deployment options as per your preference. Choose Next and Submit.
    Illustrates where to provide your Region’s preferences.
    Figure 5: Configure the Regional settings.
  7. Confirm that the StackSet creates successfully.
    Illustrates successful Stackset completion with a status of CURRENT.
    Figure 6: StackSet creation confirmation.
  8. Create a Tag Policy in the AWS Management account.
    Illustrates how to create Tag policies from the AWS Organizations console.
    Figure 7: Tag policy creation.
  9. Provide the name for the tag policy, and fill in the Policy Description. Fill in the Tag Key and Value as specified in Tag Schema in Step 6.
    Shows Tag key to enter, and how to enforce compliance by specifying allowed values for the key.
    Figure 8: Tag Key and value entry.
  10. Specify the resources in scope for this policy. For this use case, you’ll be enforcing it on ec2:instance.Illustrates how to enforce the tag compliance for the ec2:instance resource type.Figure 9: Specify the resources in scope for this policy.
  11. Once the policy is created, choose Attach to associate the policy to the OU or account.
  12. Create or reuse an Amazon EC2 instance with Apache webserver with the Amazon CloudWatch Agent installed. You can use the template from the prerequisite section to provision the resource. Navigate to the workload account and add tags to the instance as specified in the Tag Schema above.
    • Key: CW-Dashboard
    • Value: Apache
  13. Once the instance is tagged with the above tag schema, the tagHandler Lambda fetches the instance id and stores it in the AWS Systems Manager parameter store. The dashboardHandler lambda fetches the instance and creates widgets in Amazon CloudWatch Dashboards.
    Illustrates the CloudWatch dashboard with various widgets with key performance metrics.
    Figure 10: An example Amazon CloudWatch dashboard.

Clean up

  1. Navigate to your Management account → CloudFormation → StackSets
  2. Choose the Stackset created in Step 1.a
  3. Choose Actions → ‘Delete Stacks from StackSets’
  4. To delete the Stackset instances, follow the same steps as 4.a and 5.a
  5. To delete the Tag Policy, navigate to AWS Organizations → Policies → Tag policies → CW-Dashboard
  6. Choose Delete, enter CW-Dashboard to confirm deletion, and then choose Delete

Conclusion

In this post, you learned how to leverage CloudFormation StackSets and AWS Organizations tag policy to deploy the CloudWatch Monitoring Framework in a multi-account environment, with tagging enforcement. Try this solution to automate setting up CloudWatch dashboards for your workloads.

To learn more about Amazon CloudWatch Monitoring Framework, visit the GitHub repository.

Authors:

Kingston Bosco

Kingston Bosco is a Senior DevOps Consultant with Amazon Web Services. He builds solutions that help customers improve DevOps tools and processes. He holds a Masters in Information Systems. In his spare time he likes hiking with his dogs and playing soccer.

Kishore Dhamodaran

Kishore Dhamodaran is a Senior Solutions Architect at AWS. Kishore helps strategic customers with their cloud enterprise strategy and migration journey, leveraging his years of industry and cloud experience.

Jared Keating

Jared Keating is a Senior Cloud Consultant with Amazon Web Services Professional Services. Jared assists customers with their cloud infrastructure, compliance, and automation requirements drawing from his over 20 years of experience in IT.