How can I avoid DNS resolution failures with an Amazon EC2 Linux Instance?

Last updated: 2020-04-13

How can I avoid DNS resolution failures with Amazon Elastic Compute Cloud (Amazon EC2) Linux?

Short Description

To decrease CPU and network usage and avoid DNS resolution failures, apply a DNS cache.

When you use a DNS cache to query external DNS resources, such as one of the following AWS applications, most of the recurring DNS queries are answered locally by the cache without interacting with the DNS resolver over the network:

  • Amazon Relational Database Service (Amazon RDS)
  • Amazon ElastiCache
  • Amazon Simple Storage Service (Amazon S3)

The following procedure applies to all versions of Amazon Linux. If you're using another distribution, select the documentation for your distribution from the following list:

Resolution

Set up a local DNS cache, using dnsmasq (a DHCP and cache DNS server). For an overview of dnsmasq, see the DNSMASQ documentation.

1.     Install the dnsmasq server by running the following command:

sudo yum install -y dnsmasq

2.     Create a dedicated system user to run dnsmasq using the following commands:

sudo groupadd -r dnsmasq
sudo useradd -r -g dnsmasq dnsmasq

Note: dnsmasq typically runs as the root user, but drops root privileges after startup by changing to another user (by default, the user is "nobody").

3.     Create a copy of the dnsmasq.conf file using the following command:

sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.orig

4.     Open the configuration file using a text editor (for example, vim):

sudo vim /etc/dnsmasq.conf

5.     Edit the /etc/dnsmasq.conf file so that it is similar to the following:

# Server Configuration
listen-address=127.0.0.1
port=53
bind-interfaces
user=dnsmasq
group=dnsmasq
pid-file=/var/run/dnsmasq.pid

# Name resolution options
resolv-file=/etc/resolv.dnsmasq
cache-size=500
neg-ttl=60
domain-needed
bogus-priv

6.     Create the /etc/resolv.dnsmasq file, and then set the Amazon DNS server or the custom domain-name-servers that you have specified on DHCP Options Sets:

sudo bash -c "echo 'name server 169.254.169.253' > /etc/resolv.dnsmasq"

Note: For EC2-Classic, the Amazon DNS server is located at 172.16.0.23. For EC2-VPC, you can find more information about DNS server locations at DHCP Options Sets. If you create an AMI from an instance with the dnsmasq cache to launch in another VPC with a different CIDR, or if you have a custom DNS server specified in your DHCP options, then adjust the file /etc/resolv.dnsmasq to use the nameserver for that network.

7.     Restart the dnsmasq server and set the service to start up on boot using the following commands:

Amazon Linux 1

sudo service dnsmasq restart
sudo chkconfig dnsmasq on

Amazon Linux 2

sudo systemctl restart dnsmasq.service
sudo systemctl enable dnsmasq.service

8.     Verify that dnsmasq is working correctly using the dig command:

dig aws.amazon.com @127.0.0.1

If the response is similar to the following, then the dnsmasq cache is working correctly:

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.62.rc1.56.amzn1 <<>> aws.amazon.com @127.0.0.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 25122
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;aws.amazon.com.            IN    A

;; ANSWER SECTION:
aws.amazon.com.        41    IN    A    54.239.31.69

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
...

9.     Set the dnsmasq DNS cache as the default DNS resolver.

Note: You must suppress the default DNS resolver provided by DHCP by changing or creating the /etc/dhcp/dhclient.conf file. For more information, see My private Amazon EC2 instance is running Amazon Linux, Ubuntu, or RHEL. How do I assign a static DNS server to the EC2 instance that persists during reboot?

10.    Configure the default DNS resolver as a fallback option by using the following commands:

sudo bash -c "echo 'supersede domain-name-servers 127.0.0.1, 169.254.169.253;' >> /etc/dhcp/dhclient.conf"

11.    To apply the change, run the dhclient command, or reboot your instance:

sudo dhclient

- or -

sudo reboot

To verify that your instance is using the DNS cache, run the dig command:

dig aws.amazon.com

If the response indicates that the server replying to your DNS request is 127.0.0.1, then the DNS cache is working correctly:

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.62.rc1.56.amzn1 <<>> aws.amazon.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1028
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;aws.amazon.com.            IN    A

;; ANSWER SECTION:
aws.amazon.com.        55    IN    A    54.239.31.69

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1) <<<-------
...

Automation

You can use one of the following to automate the installation and configuration of dnsmasq as a DNS resolver on Amazon Linux:

If you want to automate the dnsmasq installation on other Linux distributions, you can use either file to make the necessary customization.

Both files can be run on VPC instances or EC2-Classic because they use the Amazon DNS server alternative address of 169.254.169.253 for VPC and 172.16.0.23 for EC2-Classic.

Either file can be run at launch time by passing the contents of the file in the user data field. The Bash script can be run as a standalone script or with an AWS Systems Manager Run Command to perform the actions on an existing instance.

To run the Bash script as a standalone script:

1.    Download the script on your instance and make it executable.

wget https://raw.githubusercontent.com/awslabs/aws-support-tools/master/EC2/AutomateDnsmasq/AutomateDnsmasq.sh
chmod +x AutomateDnsmasq.sh

2.    Run the following command as a root user or use sudo.

sudo ./AutomateDnsmasq.sh

Did this article help you?

Anything we could improve?


Need more help?