How do I mount Amazon EFS using the EFS DNS name on a Linux Machine that is joined with AWS Managed Microsoft AD?

Short description

When you join your Linux machine with AWS Managed Microsoft AD, you configure your instance to use the DNS servers for your Active Directory.

For AWS Managed Microsoft AD, all DNS requests are forwarded to the IP address of the Amazon provided DNS servers for your VPC. These DNS servers resolve names that are configured in your Amazon Route 53 private hosted zones. If you aren't using Route 53 private hosted zones, then your DNS requests are forwarded to public DNS servers. If no private hosted zone exists for your AWS services, then DNS requests are forwarded to public DNS servers. This means that they can only resolve AWS services DNS to public IP addresses. For more information, see Configure DNS.

With Amazon EFS, the file system DNS name automatically resolves to the mount target’s IP address in the Availability Zone of the connecting Amazon EC2 instance. This is a private IP address, which can only be resolved within the same VPC. By changing the DNS servers from the defaulted VPC-provided DNS, the EFS is no longer able to resolve the IP address so mounting by DNS fails. For more information, see Mounting on Amazon EC2 with a DNS name.

Example of issue

This example uses the AWS Managed Microsoft AD. The DNS servers provided are 172.31.28.100 and 172.31.4.147. The EFS file system was created in the same VPC with mount target 172.31.47.69.

1.    Using netcat, confirm that the EC2 instance can establish a connection with the EFS mount target 172.31.47.69:

$ nc -vz 172.31.47.69 2049
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 172.31.47.69:2049.
Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds.

2.    On the EC2 Linux server, you can mount the EFS using the DNS name. The EFS is then unmounted.

sudo mount -t efs -o tls fs-123456:/ efs
df -Th
Filesystem     Type      Size  Used Avail Use% Mounted on
devtmpfs       devtmpfs  475M     0  475M   0% /dev
tmpfs          tmpfs     483M     0  483M   0% /dev/shm
tmpfs          tmpfs     483M  516K  483M   1% /run
tmpfs          tmpfs     483M     0  483M   0% /sys/fs/cgroup
/dev/xvda1     xfs       8.0G  1.6G  6.5G  19% /
tmpfs          tmpfs      97M     0   7M   0% /run/user/0
tmpfs          tmpfs      97M     0   97M   0% /run/user/1000
127.0.0.1:/    nfs4      8.0E     0  8.0E   0% /home/ec2-user/efs
sudo umount /efs

3.    The /etc/resolv.conf file shows the Amazon provided DNS and nameserver:

cat /etc/resolv.conf 
        ; generated by /usr/sbin/dhclient-script
        search eu-west-2.compute.internal
        options timeout:2 attempts:5
        nameserver 172.31.0.2

4.    On the EC2 Linux server, integrate Microsoft AD, and then configure the Active Directory DNS servers:

echo 'supersede domain-name-servers 172.31.28.100, 172.31.4.147;' | sudo tee --append /etc/dhcp/dhclient.conf
echo 'supersede domain-search "rachel.com";' | sudo tee --append /etc/dhcp/dhclient.conf
sudo dhclient -r
sudo dhclient

5.    Confirm that the DNS servers are configured by checking the resolv.conf file:

cat /etc/resolv.conf 
options timeout:2 attempts:5
; generated by /usr/sbin/dhclient-script
search rachel.com. eu-west-2.compute.internal
nameserver 172.31.28.100
nameserver 172.31.4.147

6.    Run dig on the file system to see that the mount target private IP isn't returned:

$ dig fs-123456.efs.eu-west-2.amazonaws.com 
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.amzn2.5.2 <<>> fs-123456.efs.eu-west-2.amazonaws.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 57378
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;fs-123456.efs.eu-west-2.amazonaws.com. IN A

Note: The DNS request doesn't resolve to any A record, and the status shows NXDOMAIN.

7.    Mounting the EFS using DNS name fails:

sudo mount -t efs -o tls fs-123456:/ efs
Failed to resolve "fs-123456.efs.eu-west-2.amazonaws.com" - check that your file system ID is correct, and ensure that the VPC has an EFS mount target for this file system ID.
See https://docs.aws.amazon.com/console/efs/mount-dns-name for more detail.
Attempting to lookup mount target ip address using botocore. Failed to import necessary dependency botocore, please install botocore first.

If you use the Amazon provided name server for the VPC, then note that it successfully resolves:

dig @172.31.0.2 fs-123456.efs.eu-west-2.amazonaws.com
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.amzn2.5.2 <<>> fs-123456.efs.eu-west-2.amazonaws.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24926
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;fs-123456.efs.eu-west-2.amazonaws.com. IN A

;; ANSWER SECTION:
fs-123456.efs.eu-west-2.amazonaws.com. 60 IN	A 172.31.25.79

Resolution

Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.

Configure conditional forwarders for your Microsoft AD to forward requests to the Amazon VPC-provided DNS. This method also works for resolving other AWS services DNS to their private IP addresses, if you use the Active Directory provided DNS.

To do this, use the AWS CLI command to create a conditional forwarder rule. This forwards all sub-domains of a domain to a specific DNS server IP. For example, you can forward all DNS requests for sub-domains of amazonaws.com to the private IP of the Amazon VPC provided DNS.

Note: The Amazon VPC provided DNS IP is the reserved IP address at the base of the VPC IPv4 network range plus two.

To create the conditional forwarder rule, run the AWS CLI command create-conditional-forwarder on the command line of the Linux instance that you want to mount the EFS on:

aws ds create-conditional-forwarder --directory-id d-9c671fb35f --remote-domain-name amazonaws.com --dns-ip-addrs 172.31.0.2 --region eu-west-2

Use the following parameters:

• directory-id - Enter the AD directory ID.
• remote-domain-name - You can specify any domain. This rule is applied to all FQDN matching this domain or sub-domains.
• dns-ip-addrs - Enter the Amazon VPC provided DNS IP.

This allows for DNS resolution of the EFS DNS and the EFS can now be mounted with the DNS name.

dig fs-123456.efs.eu-west-2.amazonaws.com 

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.amzn2.5.2 <<>> fs-123456.efs.eu-west-2.amazonaws.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24926
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;fs-123456.efs.eu-west-2.amazonaws.com. IN A
;; ANSWER SECTION:
fs-123456.efs.eu-west-2.amazonaws.com. 60 IN	A 172.31.25.79

The EFS can now be mounted using the DNS name.

sudo mount -t efs -o tls fs-123456:/ efs
[ec2-user@ip-172-31-35-167 ~]$ df -Th
Filesystem     Type      Size  Used Avail Use% Mounted on
devtmpfs       devtmpfs  475M     0  475M   0% /dev
tmpfs          tmpfs     483M     0  483M   0% /dev/shm
tmpfs          tmpfs     483M  520K  483M   1% /run
tmpfs          tmpfs     483M     0  483M   0% /sys/fs/cgroup
/dev/xvda1     xfs       8.0G  1.6G  6.5G  19% /
tmpfs          tmpfs      97M     0   97M   0% /run/user/0
tmpfs          tmpfs      97M     0   97M   0% /run/user/1000
127.0.0.1:/    nfs4      8.0E     0  8.0E   0% /home/ec2-user/efs

---

Related information

How do I see a list of my Amazon EC2 instances that are connected to Amazon EFS?

How can I mount an Amazon EFS volume to AWS Batch in a managed compute environment?