How can I resolve the errors that I received for the failing CloudWatch canary I created in my VPC?

Last updated: 2021-01-08

I created an Amazon CloudWatch canary in a virtual private cloud (VPC). However, I received an error and my canary failed. How can I fix this?

Short description

When you create a canary in a VPC, an AWS Lambda function with an elastic network interface is created in the specified VPC subnets. These elastic network interfaces have no public IP addresses by default, even if they were added to public subnets. To write logs to Amazon Simple Storage Service (Amazon S3) and publish data points to CloudWatch, the canary needs access to:

  • Amazon S3 endpoint
  • CloudWatch Monitoring endpoint

Possible error messages and their causes include:

  • If there's no access to CloudWatch Monitoring, then you receive the error "No test result returned. Connection timed out after 60000ms". Note that "60000ms" reflects the timeout period configured on the canary. You also receive the "No artifacts were uploaded" error, and there are no data points in canary metrics.
  • If there's no access to Amazon S3, then you receive the "No artifacts were uploaded" and "No test result returned. Connection timed out after 300000ms" errors. Note that "300000ms" reflects the timeout period configured on the canary. If there are data points in canary metrics but you receive these errors, then the Amazon S3 endpoint isn't accessible from the subnet for your canary but the CloudWatch Monitoring endpoint is accessible.
  • If a response from your monitored endpoint exceeds the timeout period configured for the page.goto method (such as "const response = await page.goto(URL, {waitUntil: 'domcontentloaded', timeout: 30000});", where timeout is 30 seconds), then you receive the "TimeoutError: Navigation Timeout Exceeded:30000ms exceeded" error.
  • If a security group, network access control list, or routing table doesn't allow access to the canary's endpoint, then you receive the "TimeoutError: Navigation Timeout Exceeded:30000ms exceeded" error. 30000ms is the timeout period configured on the page.goto function.

Resolution

Resolve "Navigation timeout" errors

  • Confirm that the security group attached to the canary:
    • Has an outbound rule that allows connections between the configured port and the IP address of the monitored endpoint.
    • Allows outbound traffic to port 443 (Amazon S3 and CloudWatch Monitoring endpoints are reachable through HTTPS). If necessary, add an outbound rule using HTTPS (TCP port 443) for Type, and 0.0.0.0/0 for Destination.
  • Confirm that the network access control list (ACL) in the canary subnet allows inbound and outbound access.
    • Inbound:
      • Allow ephemeral ports to the source address and port of your endpoint IP address.
      • Allow ephemeral ports to 0.0.0.0/0 for port 443.
    • Outbound:
      • Allow Destination address and port of your endpoint IP address.
      • Allow Destination to 0.0.0.0/0 for port 443.
  • Confirm that your endpoint is responding within the configured timeout period:
    • Find your endpoint respond time.
time curl http/(s)://[your Endpoint IP/DNS]:Port
    • In the canary code, change the heartbeat timeout to be more than your endpoint respond time.
const response = await page.goto(URL, {waitUntil: 'domcontentloaded', timeout: 30000});

Resolve "No artifacts were uploaded" or "No test result returned" errors

Add the canary in private subnets with 0.0.0.0/0 to the network address translation (NAT) gateway or NAT instance.

  1. Create a NAT gateway.
  2. Update the canary's private subnet routing table.

-or-

Add the canary in private subnets with the VPC endpoints for Amazon S3 and CloudWatch Monitoring

  1. Follow these steps to create a Gateway Endpoint for the Amazon S3 endpoint.
    Note: For Service name, search for "Amazon S3", and then select com.amazonaws.region.s3.
  2. Follow the same steps to create an Interface Endpoint for the CloudWatch Monitoring endpoint. Be sure to select the Enable DNS name check box.
    Note: For Service name, search for "monitoring", and then select com.amazonaws.[region].monitoring. Be sure to select the Enable DNS name check box.

Did this article help?


Do you need billing or technical support?