Amazon EC2 Instance FAQ

Frequently Asked Questions

What should I do if my instance is no longer responding?

What if my EBS-backed instance appears to be stuck stopping?

What if my instance appears to be stuck shutting down?

What should I do if my instance is no longer responding?

This is often caused by an issue that is specific to the internal configuration of the instance. The recovery process after an instance is not responding depends on the type of instance that you are running (EBS-backed vs. instance-store backed).

First review the console output of your instance to try to determine why the reboot affected the instance. The information in the console output can sometimes provide you with enough details to understand why your instance failed.

From the AWS Management Console:

1. Select the instance.
2. From the Instance Actions menu, select View System Log.

From the Amazon EC2 API tools:

1. Run the ec2-get-console-output command.

If you are unable to determine what happened by looking at the console output read about the two instance types below.

Instance-Store Backed Instances

Instance Recovery

Generally speaking, when an instance launched from an AMI using an instance-store backed root device fails to boot you have options other than launching a replacement instance. It is always a good idea to create backups of your working instance configurations by bundling a custom AMI after any modifications. If you’re running an instance from an AMI that downloads scripts during the boot process, you may be able to modify the scripts to correct any errors you’re seeing in the console output.

Data Recovery

Data recovery of the instance store is usually not possible, although AWS Support may be able to recover some portion of the data if the instance has not been terminated and no underlying hardware issues exist. Data recovery is not a guaranteed process though and can take days to complete, so do not rely on the possibility of data recovery by AWS Support as your sole backup strategy.

EBS-Backed Instances

Before attempting to recover an EBS-backed instance you need know if you are using the pre-attached instance store (also known as ephemeral store) that is available to EBS-backed instances. This information is important to note because data in this pre-attached instance store will be lost if take the actions described later on this page. If you are using the instance store, see the section above for more details about recovery options for data stored on the pre-attached instance store.

Instance Recovery

If your instance's root device is an EBS-backed volume, attempting to stop and re-start the instance will often solve your issue. For more information, see Stopping and Starting Instances.

It is sometimes possible to manually correct errors on the root volume of an EBS-backed instance that is not correctly booting. Manually correcting errors can be a complex process and is not recommended unless you have experience with systems administration. Some example solutions that we’ve seen customers use after analyzing the console output of the failed instance include running fsck on the volume, disabling SELinux, or correcting an error in the fstab file.

To stop and re-start your instance:

1. Stop the instance that is failing to boot correctly (do not shutdown or terminate).
2. Detach the root EBS volume.
3. Attach the volume to an existing instance in the same Availability Zone, making sure to use a secondary mount point (not root).
4. Correct the configuration on the old root volume.
5. Detach the volume and re-attach it to the original instance at its original mount point.
6. Start the instance.
7. Reattach an Elastic IP address if needed.

Re-enable IO

In some cases, your EBS volume may have IO access disabled as a safe-guard. In this situation:

1. Navigate to your EBS volumes in the management console. If IO on the volume has been disabled, the "Status Checks" column in the Volume list will be listed as "Impaired."
2. You can use the console to re-enable IO by clicking on "Enable Volume IO" in the volume detail section.
3. We recommend you verify the consistency of your data by using a tool such as fsck or chkdsk.
4. If your instance is unresponsive, depending on your operating system, resuming IO may return the instance to service.

Data Recovery

If your instance is EBS-backed and is failing due to an issue related to its internal configuration (details are often available in the console output) and a standard reboot didn’t succeed; the following steps can be followed to recover data:

1. Stop the instance that is failing to boot correctly (do not shutdown or terminate).
2. Detach the root EBS volume.
3. Attach the volume to a new replacement instance (preferably launched from a previously bundled AMI) in the same Availability Zone, making sure to use a secondary mount point (not root).
4. Copy data from the volume to the replacement instance.

Additional Resources

There are a number of resources available to customers that are interested in improving their architecture, understanding best practices, and preparing for issues.

1. More information about how to architect for instance failure can be found in the Designing Fault-Tolerant Applications in the AWS Cloud whitepaper.
2. It is always a good idea to make sure you are creating backups of your working instance configurations by bundling a custom AMI.
3. The Amazon EC2 User Guide includes a section on failure resilient application concepts.

What if my EBS-backed instance appears to be stuck stopping?

Instances usually get stuck in the “stopping” state when there is an issue with the underlying host. You may be able to work around this by forcibly stopping your instance. This can be achieved via the EC2 command line tools or via the AWS Management Console.

Command Line Tools

ec2-stop-instances [your id here] --force

AWS Management Console

Right-click on your instance and select “Stop” in the drop-down (you should see a note that a forced stop is going to be performed).

Note: In either case, you may need to issue the force stop request twice.

If you are unable to forcibly stop your instance, please let us know via the AWS Forums or by opening an AWS Support case. To help expedite the resolution to your issue, please make sure to include any steps you’ve already taken when reaching out to support.

What if my instance appears to be stuck shutting down?

Instances in the “shutting down” state for longer than usual will eventually be cleaned up by automated processes within the Amazon EC2 service. Customers are not billed for any instance hours while an instance is not in the "running" state.

If you are unable to wait for your instance to terminate, please let us know via the AWS Forums or by opening an AWS Support case. To help expedite the resolution to your issue, please make sure to include any steps you’ve already taken when reaching out to support.