How do I retrieve data in my Amazon EC2 instance if I can't connect due to an internal issue in the instance?

Last updated: 2020-03-03

I want to retrieve data from an unreachable Amazon Elastic Compute Cloud (Amazon EC2) instance. How do I do this?

Short Description

If you can't connect to your EC2 instance and need to retrieve data stored in the instance, you can stop the instance, detach the root volume, and then attach the volume to a rescue instance.

Don't perform this procedure if your EC2 instance is instance store-backed or has instance store volumes containing data. This recovery procedure requires you to stop and start your instance, which means that data on instance store volumes is lost. For more information, see Determining the Root Device Type of Your Instance.

If your instance is part of an Amazon EC2 Auto Scaling group, or if your instance is launched by services that use AWS Auto Scaling, such as Amazon EMR, AWS CloudFormation, AWS Elastic Beanstalk, and so on, then stopping the instance could terminate the instance. Instance termination in this scenario depends on the instance scale-in protection settings for your Auto Scaling group. If your instance is part of an Auto Scaling group, temporarily remove the instance from the Auto Scaling group before starting the resolution steps.

Resolution

1.    Open the Amazon EC2 console.

2.    Choose Instances from the navigation pane, and then select the impaired instance.

3.    Choose Actions, choose Instance State, and then choose Stop.

4.    In the Description tab, select the Root device, and then select the EBS ID.

5.    Choose Actions, choose Detach Volume, and then choose Yes, Detach.

6.    Verify that the Volume Status is available.

7.    Launch a new EC2 instance in the same Availability Zone as the impaired instance. The new instance becomes your "rescue" instance.

Or, you can use an existing instance that you can access, if it uses the same Amazon Machine Image (AMI) and is in the same Availability Zone as your impaired instance.

8.    After the rescue instance launches, choose Volumes from the navigation pane, and then select the detached root volume of the impaired instance.

9.    Choose Actions, and then choose Attach Volume.

10.    Select the rescue instance ID (i-xxxxx) and then enter a device name. The device name must be specified as /dev/sd[f-p].

11.    Choose Attach.

12.    Use SSH to connect to the rescue instance.

13.    Run the lsblk command to verify the device name of the attached volume.

$ lsblk

The following is an example of the output.

NAME    MAJ:MIN   RM  SIZE RO TYPE MOUNTPOINT
xvda    202:0     0   15G  0 disk
└─xvda1 202:1     0   15G  0 part /
xvdf    202:80    0  15G  0 disk
    └─xvdf1 202:0 0  15G  0 part

The device name might be /dev/xvdf if you specify /dev/sdf in step 10.

14.    Create a mount directory and then mount with root privileges.

$ sudo mount -o nouuid /dev/xvdf1 /mnt

15.    Check the logs in the mount directory. If necessary, upload logs to Amazon Simple Storage Service (Amazon S3) using the AWS Command Line Interface (AWS CLI) or copy them to your local PC with the scp command.

$ sudo cp /mnt/var/log/messages .
$ sudo chmod 644 messages
$ exit

16.    After reviewing the logs, run unmount /mnt to detach the attached volume.

$ sudo umount /mnt

17.    Attach the volume to the original instance. The device name is /dev/xvda.


Did this article help you?

Anything we could improve?


Need more help?