How do I see a list of my Amazon EC2 instances that are connected to Amazon EFS?
Last updated: 2019-11-07
I want to see a list of my Amazon Elastic Compute Cloud (Amazon EC2) instances that have mounted an Amazon Elastic File System (Amazon EFS). How do I do that?
Traffic on the elastic network interface of each Amazon EFS mount target is tracked using the VPC Flow Logs. The flow logs can be pushed to Amazon CloudWatch Logs. Using CloudWatch Logs Insights, the traffic flow on the mount target's elastic network interface is filtered to provide the list of Amazon EC2 instances that have mounted an Amazon EFS in a specific timestamp.
Perform the following steps once. After completing these steps, each time you want to list the IP addresses of the clients mounting the Amazon EFS, run a query to create a current list.
1. Create a log group:
Open the CloudWatch console, and then select Logs.
Select the Actions menu, and then select Create Log Group.
Enter a Log Group Name, and then select Create Log Group.
2. Create an Identity and Access Management (IAM) role with permission for publishing flow logs to CloudWatch Logs:
Open the IAM console, and then create a new IAM role.
The IAM policy that's attached to your IAM role must include the permissions to publish the VPC flow logs to CloudWatch and must have a trust relationship that allows the flow logs service to assume the role.
3. Get the list of elastic network interfaces used by the mount target of your Amazon EFS:
Note: Amazon EFS will have a different mount target for each Availability Zone.
Open the Amazon EFS console.
Select the specific Amazon EFS and note the Network Interface ID for each mount target.
4. Create the flow logs:
Open the Amazon EC2 console, and then select Network Interfaces.
Select all the elastic network interfaces that you noted in step 3 that are being used by the mount target.
Select the Actions menu, and then select Create flow log. Use the following values when creating the flow log:
Filter: Select All
Destination: Select Send to CloudWatch Logs
Destination log group: Choose the log group created in Step 1.
IAM role: Choose the IAM Role created in Step 2.
Monitor the flow log status by selecting the specific elastic network interface that you created a flow log for. At the bottom of the screen, select Flow Logs. Verify that the Status is Active.
The first flow log should be pushed to CloudWatch Logs in about 10 minutes.
5. Verify that the flow logs are in CloudWatch Logs:
Open the CloudWatch console, and then select Logs.
Select the Log Group created in step 1.
Verify that all the log streams created in step 4 appear. Each elastic network interface has a different log stream.
6. Run a query:
To run a query in CloudWatch Logs Insights:
In the CloudWatch console, select Insights.
Select the log groups created in step 1 from the drop-down menu.
Select the duration that you want to review the flow logs for (Last 15 min, 30 min, 1 hr).
Enter the following query:
filter dstPort="2049" | stats count(*) as FlowLogEntries by srcAddr | sort FlowLogEntries desc
The above query reviews all the flow logs generated for all the mount targets and filters the logs that had the destination port set to Port=2049 (Amazon EFS clients connect to the mount targets on NFS port 2049 ). All the unique source IPs (Amazon EFS client IPs) are retrieved and sorted by the most active client connections. Activity is determined by the number of entries in the flow log.
Select Run Query. The output contains the list of private IPs of all the Amazon EC2 instances where you mounted Amazon EFS.
The following is an example of the query output:
# srcAddr FlowLogEntries 1 172.31.12.60 78 2 172.31.57.233 36 3 172.31.53.144 33 4 172.31.74.79 30 5 172.31.23.86 26 6 172.31.63.215 25
To run a query from the AWS Command Line Interface (AWS CLI):
After the VPC flow log is set up, you can use an AWS CLI command to run the query.
Verify that the AWS CLI is updated to the latest version:
$ pip install --upgrade awscli
Verify that jq is installed:
yum install -y jq
Use the following AWS CLI query using these query parameters:
log-group-name: Enter the log group name you created in step 1.
start-time / end-time: These values are in Unix/Epoch time. Use the converter found at epochconverter.com to convert human-readable timestamps to Unix/Epoch time.
test.json: You can optionally change the json file name each time you run this command. Changing the name makes sure that the previous output isn't merged with the new output.
sleep: This value (in seconds) is used as delay while the CloudWatch Logs Insights query is executed. The value entered depends on how long you want to review the flow logs. If you want to review the logs for a longer duration, such as weeks, then increase the sleep time.
aws logs start-query --log-group-name EFS-ENI-logs --start-time 1549002413 --end-time 1553063276 --query-string "filter dstPort="2049" | stats count(*) as FlowLogEntries by srcAddr | sort FlowLogEntries desc" >> test.json && sleep 10 && jq .queryId test.json | xargs aws logs get-query-results --query-id