How can I configure EC2 instances in an Auto Scaling group that has no internet access to send metrics and logs to CloudWatch?
Last updated: 2020-05-21
How can I configure Amazon Elastic Compute Cloud (Amazon EC2) instances in an Amazon EC2 Auto Scaling group that has no internet access to send logs and metrics to Amazon CloudWatch?
- Install the CloudWatch agent on an Amazon EC2 instance. This instance must have internet connectivity. Or, you can choose an Amazon EC2 instance that's already pushing the logs and metrics to CloudWatch using the CloudWatch agent.
- Verify that the CloudWatch agent is pushing metrics and logs from your Amazon EC2 instance.
- Create an Amazon Machine Image (AMI) of your Amazon EC2 instance for the Auto Scaling group.
- Create a launch template for the Auto Scaling group using the AMI you created in step 3. To enable the instances to push the metrics and logs to CloudWatch, provide the correct AWS Identity and Access Management (IAM) role in the launch template. Optionally, you can launch an Amazon EC2 instance in a public subnet from this launch template to confirm that the CloudWatch agent is pushing the required metrics and logs.
- Add interface virtual private cloud (VPC) endpoints for CloudWatch monitoring and Amazon CloudWatch Logs to the VPC that hosts the private subnets. To find the correct endpoint, see Amazon CloudWatch endpoints and quotas. Refer to "Considerations for creating an interface VPC endpoint" later for more details.
- Update the endpoint policy for each of the VPC interface endpoints you created in step 5:
Open the Amazon VPC console.
Choose Endpoints, and then select your interface endpoint.
Note: You must complete these steps for each of your VPC interface endpoints.
Choose Actions, and then choose Edit policy.
For Policy, choose Full Access.
- Create an Auto Scaling group (with private subnets enabled) using the launch template you created in step 4. The CloudWatch agent runs in the instances that you launch in this Auto Scaling group. The agent also sends metrics and logs through the VPC interface endpoints that you created in step 5.
Considerations for creating an interface VPC endpoint
- Be sure to use the endpoint that corresponds with the AWS Region of your Auto Scaling group. For example, if the Auto Scaling group is in the London Region, the endpoint for metrics is monitoring.eu-west-2.amazonaws.com. The endpoint for logs in this scenario is logs.eu-west-2.amazonaws.com.
- Confirm that you’ve enabled the Enable Private DNS name option. You can enable this option only if the Enable DNS hostnames and Enable DNS Support attributes are set to true for the VPC. If this option is disabled, the VPC interface endpoint isn't mapped to the service endpoint. As a result, the instances can't reach the public service endpoint. Enabling this option maps the service endpoint to the VPC interface endpoint and makes communication to the service endpoint private. By default, the CloudWatch agent connects to this endpoint. You can use the endpoint_override parameter in the agent configuration file to override the default endpoint, if required.
- Confirm that the rules for the security group allow communication between the endpoint network interface and the resources in your VPC that communicate with the service. The API calls for pushing the logs and metrics are HTTPS-based GET/POST requests. The endpoint network interface security group requires inbound rules for HTTPS protocol from the source IPs. The source IP addresses are the IP addresses of the EC2 instances pushing the metrics and logs, or the VPC CIDR.
- When instances are part of an Auto Scaling group, specify one of the dimensions as the Auto Scaling group name in the agent configuration file. To find the name of the Auto Scaling group, the agent gets the tags associated with the instance from the Amazon EC2 endpoint. You must add the VPC interface endpoint for the Amazon EC2 service. The agent gets the ImageId, InstanceId, and InstanceType values from the Amazon EC2 instance's metadata.