Getting started with open source Amazon CloudWatch Agent

We recently announced that we open sourced the Amazon CloudWatch Agent. Our customers have increasingly requested that we open the agent to allow community contribution, enable customization for client-specific use cases, and provide greater trust and security through design and implementation transparency. In response to these requests, we’ve made the source code for CloudWatch Agent available on GitHub under the MIT license. Developers can now contribute to the development of the agent by making suggestions, reporting issues, and submitting pull requests.

At the same time, we’ve improved the getting-started experience by adding CloudWatch Agent to the Amazon Linux 2 repository. Including CloudWatch Agent with Amazon Linux 2 provides a simple one-line installation and default configurations on Linux hosts.

In this article, we provide a brief overview of use cases for CloudWatch Agent and walk through how to install and begin configuring the agent. CloudWatch Agent is simple to install and includes default settings that will work for many users. It also gives advanced users granular control over how data is collected and sent to CloudWatch. The CloudWatch Agent is a powerful tool for collecting monitoring and operational data.

Overview

CloudWatch Agent is a software package that autonomously and continuously runs on your servers. Using CloudWatch Agent, we can collect metrics and logs from Amazon Elastic Compute Cloud (Amazon EC2), hybrid, and on-premises servers running both Linux and Windows. CloudWatch Agent provides access to more system level and in-guest metrics, in addition to host metrics already provided by Amazon EC2. The agent also lets us collect, aggregate, and summarize metrics and logs from containerized applications and microservices.

CloudWatch Agent supports a modular plugin model that developers can leverage to extend the core agent framework.

Metrics and logs are sent to Amazon CloudWatch for storage and monitoring. The CloudWatch Agent enables additional insight into workloads by capturing OS- and application-specific metrics and logs. In CloudWatch, we can create custom dashboards, create alarms that performs actions, and troubleshoot issues using Amazon CloudWatch Logs.

Installing CloudWatch Agent

The agent can be installed on Linux, Windows, and other supported operating systems by downloading the agent package from Amazon Simple Storage Service (Amazon S3), using AWS Systems Manager, AWS CloudFormation, or by installing it manually using the command line.

With the agent in the Amazon Linux 2 repository, the agent package can be installed on Linux hosts in a single step using the yum package manager:

$ sudo yum amazon-cloudwatch-agent install

On Windows Server systems, installation can be done by downloading the agent installer from Amazon S3 and running the installer:

PS C:\> wget https://s3.amazonaws.com/amazoncloudwatch-agent/windows/amd64/latest/amazon-cloudwatch-agent.msi
PS C:\> .\ amazon-cloudwatch-agent.msi

Access to Amazon Web Services resources requires permissions. Before starting the CloudWatch Agent, we must grant permissions that the agent needs to write metrics and logs to CloudWatch. Step-by-step instructions for installing the agent and granting permissions can be found in the Amazon CloudWatch Users Guide.

Getting started with configuration

The fastest, and by far simplest, way to get up and running with CloudWatch Agent is to start with the default configuration:

$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a start -c default

PS C:\> cd "C:\Program Files\Amazon\AmazonCloudWatchAgent"
PS C:\Program Files\Amazon\AmazonCloudWatchAgent> .\amazon-cloudwatch-agent-ctl -a start -c default

The default configuration will emit two metrics, mem_used_percent and disk_used_percent to CloudWatch in the CWAgent namespace. Starting the agent with a default configuration is a good way to quickly ensure that the agent is installed, that it has the correct permissions, and that we can find our metrics in the CloudWatch console.

Once we have the agent installed and are comfortable starting it, we can begin to add additional metrics and logs using the agent configuration wizard:

$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

PS C:\Program Files\Amazon\AmazonCloudWatchAgent> .\amazon-cloudwatch-agent-config-wizard.exe

Through a series of questions, the configuration wizard will select the most common options from various host metrics categories including CPU, disk, memory, and network. We can also specify any log files to send to CloudWatch. The wizard will create a configuration file we can immediate use with the agent.

The following command will start the agent using the configuration file written by the configuration wizard:

$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json –s

PS C:\Program Files\Amazon\AmazonCloudWatchAgent> .\amazon-cloudwatch-agent-ctl -a fetch-config -c file:C:\ProgramData\Amazon\AmazonCloudWatchAgent\config.json -s

Understanding the configuration file

Now is a good time to inspect the configuration file produced by the wizard, as it will already contain many of the key features that we will be using to customize the agent. The agent is a powerful package with lots of optional configurations for power users. Let’s take a look at a handful of parameters to get started.

The JSON configuration file is divided into three sections: agent, metrics, and logs. The agent section includes fields for the overall configuration of the agent, such as the metric collection frequency, and the region where the metrics should be sent. This section optionally includes parameters for IAM role and debugging and logging options:

"agent": {
  "metrics_collection_interval": 30,
  "region": " us-west-1"
}

In the metrics section, you can add individual host metrics to publish to CloudWatch. There are dozens of metrics available either on Linux or Windows instances. You can roll up metrics along single or multiple dimensions allowing you to monitor individual server metrics while at the same time having an overview of your entire fleet.

"metrics": {
       "namespace": "CWAgent",
       "metrics_collected": { 
              "cpu": {
                     "measurement": [
                           "cpu_usage_iowait",
                           "cpu_usage_user",
                           "cpu_usage_system"
                     ]
              },
             "disk": {
                     "measurement": [
                           "used_percent"
                     ]
              },
       },
       “append_dimensions”: {
              "InstanceId": "${aws:InstanceId}",
              "InstanceType":"${aws:InstanceType}"
       },
       “aggregation_dimensions”:[
          ["InstanceId", “InstanceType”],
          [“InstanceType”],
          []
       ]
}

Also in the metrics section, agent supports procstat, StatsD, and collectd plugins, which are especially useful for instrumenting our own metrics. StatsD and collectd are popular open source solutions with plugins that can gather system statistics for a wide variety of applications. The procstat plugin lets us collect metrics from individual processes. With the procstat plugin, the CloudWatch Agent can even monitor itself:

"metrics": {
  "metrics_collected": {
  "procstat": [
    {
      "exe": "amazon-cloudwatch-agent",
      "measurement": [
        "cpu_usage",
        "memory_rss"
        ]
      }
    ]
  }
}

The logs section specifies which log files are published to CloudWatch Logs. We can send log events written by any application or service. CloudWatch can extracted custom metrics data from logs using CloudWatch Embedded Metric Format. We can then easily view logs, search them for specific error codes or patterns, filter them based on specific fields, or archive them securely for future analysis:

"logs": {
  "logs_collected": {
    "files": {
      "collect_list": [
        {
          "file_path": /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log ",
          "log_group_name": "CWAgent",
          "log_stream_name": "{instance_id}"
        }
      ]
    }
  }
}

Where to go from here?

This brief introduction to the CloudWatch Agent provides a starting point for more advanced and customized configurations. A complete description of the possible configurations is available in the CloudWatch Users Guide. For many users, the default settings will create the visibility needed to start monitoring their servers and applications quickly. CloudWatch Agent also provides advanced users with the tools to collect monitoring and operational data in a way that best suits unique use cases.

Using CouldWatch Agent to send monitoring and operational data to CloudWatch in the form of logs and metrics provides a unified view of servers and applications that run on AWS and on-premises.