How do I troubleshoot Route 53 geolocation routing issues?

Last updated: 2021-05-26

My DNS queries return an IP address of a web server in a different AWS Region. For example, a user in the United States is being routed to an IP address of a web server located in Europe. How do I troubleshoot and resolve Amazon Route 53 geolocation routing issues?

Short description

Route 53 geolocation routing issues can be caused by:

  • Missing default location in your geolocation routing setup
  • DNS resolver that doesn't support the edns-client-subnet extension of EDNS0 (leads to inaccurate determination of your location)
  • Geographically diverse DNS resolvers
  • DNS changes for resource records that haven't propagated globally

Resolution

1.    Confirm that the resource records for your Route 53 hosted zone are properly configured for your use case, and that there's a default resource record set. From the Route 53 console, check the default location specified in your Route 53 hosted zone configuration.

Consider the following sample output:

>> dig images.example.com
            
; <<>> DiG
9.8.2rc1-RedHat-9.8.2-0.37.rc1.45.amzn1 <<>> images.example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR,
id: 51385
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1,
ADDITIONAL: 0

;; QUESTION SECTION
;images.example.com.                   IN            A

;; AUTHORITY SECTION:
images.example.com.      60          IN           SOA        ns-1875.awsdns-42.co.uk.awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400

;; Query time: 65 msec
;; SERVER: 172.31.0.2#53(172.31.0.2)
;; WHEN: Tue Feb  7 22:02:30 2017
;; MSG SIZE  rcvd: 124

If there isn't a default location specified in your geolocation routing configuration, the DNS response returns NOERROR for the rcode field. In this case, there's no result in the ANSWER section. To correct this issue, add a default location in your geolocation routing configuration.

2.    To check the IP address range of the DNS resolver, run the following commands 5 or 6 times for a period of 11 seconds. Be sure to note the output each time.

On Linux or macOS, use dig:

for i in {1..10}; do dig +short resolver-identity.cloudfront.net; sleep 11; done;

On Windows, use nslookup:

nslookup resolver-identity.cloudfront.net

3.     Check if the DNS resolver supports EDNS0-Client-Subnet using one of the following commands. Be sure to note the output.

On Linux or macOS, use dig:

dig +nocl TXT o-o.myaddr.l.google.com

On Windows, use nslookup:

nslookup -type=txt o-o.myaddr.l.google.com

Review the first TXT record returned in the Answer section of the output. The first TXT record value is the IP address of the DNS resolver. If there isn't a second TXT record, the DNS resolver doesn't support EDNS0-Client-Subnet. If there's a second TXT record, the DNS resolver supports EDNS0-Client-Subnet. The resolver provides and then sends a truncated client subnet IP address (/24 or /32) to the Route 53 authoritative name server.

4.    Use the Route 53 test record set from the checking tool to determine which resource records are returned for a specific request. For more information, see Using the checking tool to see how Amazon Route 53 responds to DNS queries.

If the DNS resolver doesn't support EDNS0-Client-Subnet, specify the DNS Resolver IP address as your value in the tool.

If the DNS resolver supports EDNS0-Client-Subnet, specify the EDNS0 client subnet IP address as your value in the tool. Choose More Options, and then specify the Subnet mask. Don't specify a Resolver IP address.

5.    (Optional) If you don't have access to the checking tool, use dig to query the Route 53 authoritative name servers for your hosted zone with EDNS0-Client-Subnet. Use the output to determine the lowest latency AWS Region from your source IP address:

dig geo.example.com +subnet=<Client IP>/24 @ns-xx.awsdns-xxx.com +short

6.    Route 53 supports the edns-client-subnet extension of EDNS0.The recursor or local name server appends edns-client-subnet to the DNS query to make a DNS lookup on the client's source IP subnet. If this data isn't passed with the request, Route 53 uses the source IP address of the DNS resolver to approximate the location of the client. Then, Route 53 responds to geolocation queries with the DNS record for the resolver's location. If the EDNS data isn't passed to Route 53 and the client is using a geographically diverse recursive name server, the result is a suboptimal location serving the incorrect resource record to the DNS query.

To fix this configuration, change the recursive DNS server that supports edns-client-subnet. Perform the DNS resolution, and then share the output. If the recursive DNS server doesn't support the EDNS client subnet, try using one that does. For example, options include that support EDNS include Google DNS, OpenDNS, and Amazon DNS server. For EC2-Classic, the Amazon DNS server is located at 172.16.0.23. For EC2-VPC, the Amazon DNS server is located at the base of your VPC network range plus two.

7.     Check the geographic location of the client subnet IP address using MaxMind’s the GeoIP database on the MaxMind website, or your preferred GeoIP database. Verify that the DNS resolver is geographically close to the client’s public IP address.

8.    Check for issues with DNS propagation using a tool such as CacheCheck on the OpenDNS site.

9.    (Optional) Determine if the geography-based routing records are associated with a Route 53 health check, and if Evaluate Target Health is enabled (for alias records). In one or both are true, then Route 53 returns the healthy endpoint that has lowest latency.

Check the status of your Route 53 health check in the Route 53 console. If Evaluate Target Health (ETH) is enabled, check the health status of the record endpoint. Route 53 considers an endpoint for a Classic Load Balancer with ETH enabled as healthy if at least one backend instance is healthy. For Application and Network Load Balancers, every target group with targets must contain at least one healthy target to be considered healthy. A target group with no registered targets is considered unhealthy. If any target group contains only unhealthy targets, the load balancer is considered unhealthy.

For example, let's say that you have records for Texas in the US for the US, North America, and all locations (Location is Default). If a query originates from Texas and the endpoint for Texas is unhealthy, then Route 53 checks the records for the US, North America, and all locations in that order until it finds a record with a healthy endpoint. If the US record is healthy, then Route 53 returns this endpoint. Otherwise, Route 53 returns a default record. If all applicable records are unhealthy, including the record for all locations, then Route 53 responds to the DNS query using the value of the record for the smallest geographic Region.

Note: Changes to aliased geolocation resource records can take up to 60 seconds to propagate.