AWS DevOps & Developer Productivity Blog

View AWS CodeDeploy logs in Amazon CloudWatch console

AWS CodeDeploy helps users deploy software to a fleet of Amazon EC2 or on-premises instances. The AWS CodeDeploy agent must be installed and configured on those instances in order for them to be used in a deployment. The agent generates three log files:

  • Agent log – Contains information about the agent’s health and overall deployment status.
  • Deployment log – Contains STDOUT, STDERR, and information specific to the user-defined scripts that run during a deployment.
  • Updater log (Linux agents) – Contains agent updater status.

It can be cumbersome to sign in to instances to view these logs, particularly when there are large number of deployment instances. This article describes a way to monitor these logs centrally using Amazon CloudWatch logs. We will focus on Linux instances. We assume you are familiar with AWS CodeDeploy and Amazon CloudWatch concepts and have completed the CodeDeploy walkthrough.

Requirements

Agent Requirements

  • The CodeDeploy agent version has to be 1.0-1.854 or later.
  • If you are setting up an existing instance with the CloudWatch agent already installed, the version has to be 1.3.8 or later.

Add required permissions to IAM instance profile

Similar to the CodeDeploy agent, the CloudWatch Logs agent uses the IAM instance profile associated with the EC2 instance. To learn more about creating the IAM instance profile, see the Create an IAM Instance Profile for Your Amazon EC2 Instances topic in the CodeDeploy documentation. You will need to attach the CloudWatchLogsFullAccess policy to this IAM instance profile to give the required permissions to the CloudWatch Logs agent.

  • Go to IAM console, and choose Roles.
  • Choose the role used to launch the deployment instances.
  • Choose Attach Policy.
  • Under Policy Name, select CloudWatchLogsFullAccess policy, and then choose Attach Policy.

Install the CloudWatch Logs agent

For new instances

  • Go to the EC2 console, and choose Launch Instance.
  • Choose the desired AMI.
  • Select the desired instance type, and then choose Next.
  • On the Configure Instance Details page, for IAM role, choose the CodeDeploy deployment instance role.
  • In Advanced Details, paste the following script in the User data area, and then choose Review and Launch.

Substitute REGION in this script with the appropriate AWS region. For e.g. us-east-1 for US East (N. Virginia) region.

#!/bin/bash 
wget https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py
wget https://s3.amazonaws.com/aws-codedeploy-us-east-1/cloudwatch/codedeploy_logs.conf
chmod +x ./awslogs-agent-setup.py
python awslogs-agent-setup.py -n -r REGION -c s3://aws-codedeploy-us-east-1/cloudwatch/awslogs.conf
mkdir -p /var/awslogs/etc/config
cp codedeploy_logs.conf /var/awslogs/etc/config/
service awslogs restart

For existing instances

Sign in to the instance and manually install the CloudWatch Logs agent by running the following commands:

Substitute REGION with the appropriate AWS region. For e.g. us-east-1 for US East (N. Virginia) region.

wget https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py
wget https://s3.amazonaws.com/aws-codedeploy-us-east-1/cloudwatch/codedeploy_logs.conf
chmod +x ./awslogs-agent-setup.py
sudo python awslogs-agent-setup.py -n -r REGION -c s3://aws-codedeploy-us-east-1/cloudwatch/awslogs.conf
sudo mkdir -p /var/awslogs/etc/config
sudo cp codedeploy_logs.conf /var/awslogs/etc/config/
sudo service awslogs restart

Alternatively, you can do a CodeDeploy deployment of our sample CloudWatch agent installation app:

  • Go to the CodeDeploy console.
  • Create a CodeDeploy application.
  • Create a CodeDeploy deployment group for your instances.
  • Deploy the following S3 revision:
s3://aws-codedeploy-REGION/cloudwatch/cloudwatch_log.zip

Substitute REGION with the appropriate AWS region. For e.g. us-east-1 for US East (N. Virginia) region.

If the deployment is successful, CodeDeploy agent logs on your instances should appear in the CloudWatch console.

View logs in the CloudWatch console

  • Go to the CloudWatch console, and choose Logs.
  • To view the CodeDeploy agent log, choose codedeploy-agent-logs.
  • To view the deployment scripts log, choose codedeploy-deployment-logs.
  • To view the CodeDeploy agent updater log, choose codedeploy-updater-logs.

Troubleshooting

If you don’t see logs in the CloudWatch console:

  • Make sure the CodeDeploy agent is running on the instance and satisfies the version requirements. To troubleshoot CodeDeploy, see this topic.
  • Make sure the CloudWatch agent is running on the instance and satisfies the version requirements. To troubleshoot CloudWatch agent operations, see this topic.

Conclusion

These steps should help you get started using CloudWatch Logs to track your detailed CodeDeploy activity. Let us know in the comments how this works for your scenarios, or reach out to us through the AWS forums if you have any technical questions.