Deploy Multi-Account Amazon CloudWatch Dashboards
Organizations building modern applications require a way to gain actionable insights into their Amazon Elastic Compute Cloud (Amazon EC2) workloads. Amazon CloudWatch is a monitoring and observability service that collects operational data from logs, metrics, and events. The service lets customers monitor your resources spread across different accounts or regions in a single view, visualize the operational data that it collects, and create customized views of the metrics and alarms for your AWS resources.
This post will demonstrate how you can quickly provision a multi-account CloudWatch dashboard by leveraging the Amazon CloudWatch Monitoring Framework and enforcing standardization via the AWS Organizations tag policy approach. Tag policies are a type of policy that can help you standardize tags across resources in your organization’s accounts. In a tag policy, you specify tagging rules applicable to resources when they are tagged. Creating CloudWatch dashboards and widgets can be straightforward, but creating and configuring new dashboards for every workload is time-consuming. Amazon CloudWatch Monitoring Framework automates the process of dashboard setup and provides reference config files for some of the most common workloads. The framework deploys from a your central management account with AWS Organizations and tagging enabled, and it will use AWS CloudFormation StackSets for multi-account deployment. CloudFormation StackSets extends the capability of stacks by enabling you to create, update, or delete stacks across multiple accounts and AWS Regions with a single operation.
In this post, you’ll automatically create CloudWatch Dashboard widgets for an Amazon EC2 instance, running a simple Apache server by tagging the Amazon EC2 instances. First, you’ll deploy the CloudWatch monitoring Framework CloudFormation template using CloudFormation StackSets to the workload account in development and staging environments. Then, you’ll deploy a stack that will create an application running on an Apache webserver.
Figure 1: Solution architecture and workflow
You’ll complete the following actions as part of the solution:
- Amazon EC2 instances are tagged following our AWS Organizations tag policy.
- tagHandler AWS Lambda function fetches instances with the tag and updates the AWS Systems Manager Parameter Store.
- Systems Manager triggers the CloudWatch Events rule, and this invokes the dashboardHandler Lambda function.
- dashboardHandler Lambda function reads the Systems Manager parameter and updates the CloudWatch dashboard widgets
The following prerequisites are required:
- A management account with AWS Organizations enabled. Review Tutorial: Creating and configuring an organization for detailed steps.
- AWS workload accounts (development and staging) for deploying the solution
- Account ID
- Deployment region
- You must create the AWS Identity and Access Management (IAM) roles required by CloudFormation StackSets to deploy across accounts and regions. These roles are necessary to establish a trusted relationship between the account that you’re administering the stack set from and the account to which you’re deploying stack instances. Refer to Grant self-managed permissions to understand the requirements for CloudFormation StackSets. There are three options to set up the permissions required to perform CloudFormation StackSet operations across multiple accounts. Review Prerequisites for stack set operations for detailed prerequisite steps for each option. For this post, we’ll be using self-managed permissions.
- Enable Tag Policies in AWS Organization. You can refer to Creating, updating, and deleting tag policies to see how to enable and work with tag policies.
- Download the CloudWatch Monitoring Framework template.
- Download the Apache workload template.
The following section will walk you through the solution steps.
- Navigate to the AWS Cloud formation console, choose StackSets from the navigation pane, then choose Create Stackset.
Figure 2: Creating the CloudFormation StackSet.
- You will use self-managed permissions. If using the same option, provide the StackSet Admin and Execution Role that were created as part of the prerequisite. Figure 3: Setting permissions for CloudFormation StackSets.
- Under Prerequisite – Prepare template, choose Template is ready. Under Specify template, choose Upload Template. Choose file and provide the template that you previously downloaded, and choose Next.
- In the next prompt, under StackSet name, enter
amazon-cloudwatch-monitoring. Fill the parameters as shown. Then choose Next.
- Workload Name:
- Tag Schema:
- CW Dashboard Name:
- Access Log Group:
- SSM Parameter Name:
- Workload Name:
- Under Deployment locations, choose Deploy stacks in accounts, and provide the workload account ids in the box.Figure 4: Configure the deployment options.
- Choose the region to which you would like to deploy the stack. Provide the deployment options as per your preference. Choose Next and Submit.
Figure 5: Configure the Regional settings.
- Confirm that the
Figure 6: StackSet creation confirmation.
- Create a Tag Policy in the AWS Management account.
Figure 7: Tag policy creation.
- Provide the name for the tag policy, and fill in the Policy Description. Fill in the Tag Key and Value as specified in Tag Schema in Step 6.
Figure 8: Tag Key and value entry.
- Specify the resources in scope for this policy. For this use case, you’ll be enforcing it on ec2:instance.Figure 9: Specify the resources in scope for this policy.
- Once the policy is created, choose Attach to associate the policy to the OU or account.
- Create or reuse an Amazon EC2 instance with Apache webserver with the Amazon CloudWatch Agent installed. You can use the template from the prerequisite section to provision the resource. Navigate to the workload account and add tags to the instance as specified in the Tag Schema above.
- Once the instance is tagged with the above tag schema, the tagHandler Lambda fetches the instance id and stores it in the AWS Systems Manager parameter store. The dashboardHandler lambda fetches the instance and creates widgets in Amazon CloudWatch Dashboards.
Figure 10: An example Amazon CloudWatch dashboard.
- Navigate to your Management account → CloudFormation → StackSets
- Choose the Stackset created in Step 1.a
- Choose Actions → ‘Delete Stacks from StackSets’
- To delete the Stackset instances, follow the same steps as 4.a and 5.a
- To delete the Tag Policy, navigate to AWS Organizations → Policies → Tag policies → CW-Dashboard
- Choose Delete, enter
CW-Dashboardto confirm deletion, and then choose Delete
In this post, you learned how to leverage CloudFormation StackSets and AWS Organizations tag policy to deploy the CloudWatch Monitoring Framework in a multi-account environment, with tagging enforcement. Try this solution to automate setting up CloudWatch dashboards for your workloads.
To learn more about Amazon CloudWatch Monitoring Framework, visit the GitHub repository.