AWS Cloud Operations Blog
Keeping CloudWatch Dashboards up to date using AWS Lambda
With the launch of the new CloudWatch Dashboards API and CloudFormation support it is now easy to automate your CloudWatch Dashboards and make sure they monitor all the resources that you launched when creating your CloudFormation stacks.
Let’s now see how you can use the new CloudWatch Dashboards API to dynamically update your dashboard as EC2 instances are added or removed. When you use auto-scaled EC2 instances for example, EC2 instances may be launched or terminated at any time and if you have a CloudWatch Dashboard monitoring your EC2 resources it can suddenly be monitoring instances that do not exist anymore, and may be missing ones that do exist.
Use AWS Lambda to keep EC2 instances up to date
A simple solution is to run the script below periodically in AWS Lambda. The script loads your CloudWatch Dashboards that monitors your instances and updates the EC2 graph widgets if needed.
The script:
- Loads the specified CloudWatch Dashboard(s)
- Looks for all graph widgets displaying EC2 instance metrics
- Calls EC2 DescribeInstances API with configured parameters to discover the current EC2 instances for that graph in that region
- Updates the widget if needed
- Saves the CloudWatch Dashboards if any widget definition has changed
The script is configured via an environment variable AWS_DASHBOARDS whose value is a JSON array of the dashboard names that you want to update along with (optional) parameters for EC2 DescribeInstances API which sets which EC2 Instances should be displayed on each dashboard.
Here’s an example that will update a CloudWatch Dashboard called MyAutoScalingDashboard
with all running instances in the AutoScaling group called MyAutoScalingGroup
:
To create the Lambda function:
- In the Lambda console, choose Create a Lambda function
- Select Blank Function and choose Next when asked to configure a trigger.
- For Name, enter
ec2DashboardUpdater
- For Code entry type choose Upload a file from Amazon S3.
- For S3 link URL enter
https://s3.amazonaws.com/ec2-dashboard-updater/ec2DashboardUpdater.zip
- You’ll need to specify the
AWS_DASHBOARDS
environment variable with a JSON array for the dashboards you want to update, e.g. with the JSON in the example earlier (no newlines: - Set the Handler as
ec2DashboardUpdater.handler
- For Role select Create a custom role
- In the IAM Console window that opens choose IAM Role of Create a new IAM Role
- Set Role Name of
ec2DashboardUpdaterRole
- Choose View Policy Document, Edit, Ok.
- Set the role to:
- Choose Allow
- In Advanced settings set Timeout to a value that is enough to describe all instances for all EC2 graph widgets and load and save all dashboards – set it to 5 min if unsure
- Select Next then Create function
To call the Lambda function periodically:
- In the CloudWatch console, choose Rules in the left navigation pane
- Select Create rule
- Choose the Schedule radio button and set Fixed rate of
15
Minutes - Select Add target, make sure Lambda function is active and for Function select the previously created
ec2DashboardUpdater
function - Choose Configure details
- Enter Name as
ec2DashboardUpdaterCron
- Select Create rule
And that’s it, you’re done. Your selected dashboards will now be kept up to date with your changing EC2 instances hourly. If your EC2 instances do not change from hour to hour then your dashboards will not be updated.
Please note some limitations of the script:
- It will only update graphs where the first metric is an EC2 instance id metric, such as CPUUtilization, NetworkIn or DiskReadBytes, in the AWS/EC2 namespace.
- It assumes EC2 metrics on a graph all use the same MetricName field.
- All EC2 graphs on a particular dashboard are for the same list of EC2 instances (as specified in the ec2DescribeInstanceParams configuration parameter.
The (Javascript) code for the script is given below. Feel free to use it as the basis for keeping your CloudWatch Dashboards up to date with other resources.