How can I monitor memory usage in Elastic Beanstalk for Amazon Linux AMIs running on Amazon EC2 instances?

Last updated: 2020-11-19

The memory usage for my Amazon Linux Amazon Machine Images (AMIs) that are running on Amazon Elastic Compute Cloud (Amazon EC2) instances in AWS Elastic Beanstalk is too high and growing. I want to monitor memory usage over time.

Short description

To monitor memory (RAM) usage, you must add a custom Amazon CloudWatch metric using a .ebextensions file. The .ebextensions file runs a script in the environments of your Amazon EC2 instances. The script periodically issues memory metrics and other custom CloudWatch metrics.

Important: The .ebextension used in the following resolution is applicable only for Amazon Linux AMI instances and not for Windows or custom Ubuntu AMI instances in Elastic Beanstalk.

Resolution

1.    Create a .ebextensions file (such as .ebextensions/cloudwatch.config) based on the following example:

packages:
  yum:
    perl-DateTime: []
    perl-Sys-Syslog: []
    perl-LWP-Protocol-https: []
    perl-Switch: []
    perl-URI: []
    perl-Bundle-LWP: []
sources: 
  /opt/cloudwatch: https://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip
  
container_commands:
  01-setupcron:
    command: |
      echo '*/5 * * * * root perl /opt/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl `{"Fn::GetOptionSetting" : { "OptionName" : "CloudWatchMetrics", "DefaultValue" : "--mem-util --disk-space-util --disk-path=/" }}` >> /var/log/cwpump.log 2>&1' > /etc/cron.d/cwpump
  02-changeperm:
    command: chmod 644 /etc/cron.d/cwpump
  03-changeperm:
    command: chmod u+x /opt/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl
option_settings:
  "aws:autoscaling:launchconfiguration" :
    IamInstanceProfile : "aws-elasticbeanstalk-ec2-role"
  "aws:elasticbeanstalk:customoption" :
    CloudWatchMetrics : "--mem-util --mem-used --mem-avail --disk-space-util --disk-space-used --disk-space-avail --disk-path=/ --auto-scaling"

Important: For Amazon Linux 2 based platforms, replace the perl-Bundle-LWP yum package in the preceding configuration file with perl-Digest-SHA.

2.     Add the following permissions to the Elastic Beanstalk environment's instance profile:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "cloudwatch:PutMetricData",
        "ec2:DescribeTags"
      ],
      "Effect": "Allow",
      "Resource": [
        "*"
      ]
    }
  ]
}

Note: The permissions in step 2 allow your environment's instances to publish the metrics in step 1 to CloudWatch. You can use AWS Identity and Access Management (IAM) conditions to create a more restrictive policy. For more information, see Condition keys for Amazon EC2.

3.    Create an application source bundle that includes your .ebextensions file from step 1.

4.    Deploy your updated Elastic Beanstalk application.

5.    View the metrics in the CloudWatch console in the Linux System namespace section.

Note: For an example of a configuration file that includes metrics, alarms, and scaling for memory usage, see an example file on AWS GitHub.