Feature Guide: Elastic Block Store

Amazon Elastic Block Storage (Amazon EBS) is a new type of storage designed specifically for Amazon EC2 instances. Amazon EBS allows you to create volumes that can be mounted as devices by EC2 instances. Amazon EBS volumes behave as if they were raw unformatted external hard drives. They have user supplied device names and provide a block device interface. You can create a file system on top of Amazon EBS volumes, or use them in any other way you would use a block device.

You can create up to twenty Amazon EBS volumes of any size (from one gigabyte up to one terabyte). Each Amazon EBS volume is independent of any individual EC2 instances and can be attached to any one of your instances (within the same availability zone) or can even exist unattached to an EC2 instance.

Amazon EBS provides the ability to create snapshots of your Amazon EBS volumes to Amazon S3. These snapshots can be used as the starting point for new Amazon EBS volumes and can protect your data for long term durability.

Amazon EBS Volumes:

• Provide off-instance storage

• Persist beyond the lifetime of instances

• Provide high availability and reliability

• Attach to and detach from a running instance

• Expose as a device within an instance

Amazon EBS Snapshots:

• Capture the current state of a volume

• Provide backup protection

• Can be used to instantiate new volumes, which will contain the exact data of the snapshot

This section describes common Amazon EBS use cases:

Amazon EBS is designed to allow you to attach a storage volume to any instance. In the event you experience an instance failure, your Amazon EBS volume automatically detaches with your data intact. You can then reattach the volume to a new instance and quickly recover.

For example:

1. You are running an Amazon EC2 instance that is attached to an Amazon EBS volume when your Amazon EC2 instance fails or is experiencing problems.
2. To recover, you detach the Amazon EBS volume from your instance (if it has not already automatically detached), launch a new Amazon EC2 instance, and attach the Amazon EBS volume to the new instance.
3. In the unlikely event the Amazon EBS volume fails, you can instantiate a new Amazon EBS volume based on the most recent snapshot you had made of your failed volume.

   Volume Instantiation Using Snapshot

Amazon EBS snapshots allow you to quickly launch new volumes, using a previous snapshot as a starting point for those new volumes.

For example:

1. You are running a web service with a large data set.
2. When the data is in a desired state, you snapshot the data to Amazon S3.
3. As traffic and resource usage increases, you can create a new volume from the snapshot, launch a new instance and then attach the new volume to the new instance.
4. If traffic decreases, you can shut down one or more Amazon EC2 instances and delete their Amazon EBS volumes.

Amazon EBS volumes exist separately from the actual instances and persist until you delete them. This allows you to retain your data without leaving an Amazon EC2 instance running.

For example:

1. You run an instance periodically to perform a batch processing job on a large and growing data set.
2. At the end of your job, you shut down the Amazon EC2 instance, but leave your Amazon EBS volume running.
3. The next time you process the data set, launch a new EC2 instance and reattach it to your existing Amazon EBS volume
4. Using this model, you can process and store your data set indefinitely, only using the processing and storage resources that you require.

Amazon EBS supports larger volumes than provided by Amazon EC2 instances. You can create up to one terabyte Amazon EBS volumes.

Before using Amazon EBS, you must download the latest version of the the EBS tools.

When completed, you should be using API Tools version 1.3-18822 and WSDL/API version 2008-02-14. You can check this by running the following command:

$ ec2ver 1.3-18822 2008-02-14

To configure and use Amazon EBS, we provide eight new API functions. This section provides a brief overview of each function.

The CreateVolume function creates a new Amazon EBS volume of the specified size or creates a new volume based on a previously created snapshot. Once the volume has been created, it is totally independent from the snapshot it was based on and changes to the volume will not affect the snapshot.

The DeleteVolume function deletes the specified volume. However, it does not delete any snapshots that were created from this volume. For information on deleting snapshots, see DeleteSnapshot.

The DescribeVolumes function describes all volumes, including their size, status (available, in-use), creation time and possible active attachment.

The AttachVolume function attaches the specified volume to a specified instance, exposing the volume using the specified device name. A volume can only be attached to a single instance at any time.

The DetachVolume function detaches the specified volume from the instance to which it is attached. This operation does not delete the volume. The volume can be attached to another instance and will have the same data as when it was detached.

Note: A volume must be unmounted inside the instance before being detached. Failure to do so will result in damage to the file system or the data it contains.

The CreateSnapshot function creates a snapshot of the specified volume. Once created, you can use the snapshot to create volumes that contain the exact same data as the original volume. The call will return after the snapshot initializes. The snapshot is completed asynchronously and progress can be monitored using DescribeSnapshots.A snapshot is totally independent from the volume. Any changes to the volume will not affect the snapshot.The time required to complete the snapshot depends on the amount of data stored on the volume and the rate at which new data is written. Snapshots are saved to Amazon S3 for durable backup. Additionally, each snapshot is compressed, with only the differential changes being stored. This helps to minimize the space consumed on S3 and lowers your snapshot storage costs.

The DeleteSnapshot function deletes the specified snapshot. This function does not affect currently running Amazon EBS volumes, regardless of whether they were used to create the snapshot or were derived from the snapshot.

The DescribeSnapshots function describes all snapshots including the volume being snapshotted, the snapshot status (pending, completed), start time and progress.

This section shows examples of how to use the APIs for Amazon EBS volumes and snapshots.

In this example, the user calls the CreateVolume API, specifying an 800 GiB volume.

$ ec2-create-volume --size -z

VOLUME vol-4d826724 800 creating 2008-02-14T00:00:00+0000

$ ec2-describe-volumes vol-4d826724

VOLUME vol-4d826724 800 available 2008-02-14T00:00:00+0000

In this example, the user calls the AttachVolume API to attach the volume vol-4d826724 to the instance i-6058a509 and expose it as the device /dev/sdh.

$ ec2-attach-volume vol-4d826724 -i i-6058a509 -d /dev/sdh

ATTACHMENT vol-4d826724 i-6058a509 /dev/sdh attaching 2008-02-14T00:15:00+0000

After creating Amazon EBS volumes and attaching them to instances, you can list them using the DescribeVolumes and the DescribeInstances functions.

To list all volumes owned by the user, including their status, the user invokes the DescribeVolumes function.

$ ec2-describe-volumes

VOLUME vol-4d826724 800 in-use 2008-02-14T00:00:00+0000

ATTACHMENT vol-4d826724 i-6058a509 /dev/sdh attached 2008-02-14T00:00:17+0000

VOLUME vol-50957039 1 available 2008-02-091T00:00:00+0000

VOLUME vol-6682670f 1 in-use 2008-02-11T12:00:00+0000

ATTACHMENT vol-6682670f i-69a54000 /dev/sdh attached 2008-02-11T13:56:00+0000

The function returns the volume ID, capacity, status (in-use or available) and creation time of each volume. If the volume is attached, an attachment line shows the volume ID, the instance ID to which the volume is attached, the device name exposed to the instance, its status (attaching, attached, detaching, detached) and when it was attached.

The user can also view volumes that are attached to running instances by using the DescribeInstances function.

$ ec2-describe-instances

RESERVATION r-e112fc88 416161254515 default

INSTANCE i-3b887c52 ami-3fd13456 ec2-67-202-27-216.compute-1.amazonaws.com domU-12-31-38-00-35-94.compute-1.internalrunning gsg-keypair 0 m1.small 2007-11-26T13:20:35+0000 vol-4d826724

RESERVATION r-e612fc8f 416161254515 default

INSTANCE i-21b63c22 ami-3fd13456 ec2-67-202-18-227.compute-1.amazonaws.com domU-12-31-38-00-39-28.compute-1.internalrunning gsg-keypair 0 m1.small 2007-11-26T13:21:51+0000 vol-6682670f,vol-50957039

Inside the instance, the Amazon EBS volume is exposed as a normal block device and can be formatted as any file system and mounted. In this example, the user creates an ext3 file system on the Amazon EBS volume and mounts it as /mnt/data-store.

$ yes | mkfs -t ext3 /dev/sdh

$ mkdir /mnt/data-store

$ mount /dev/sdh /mnt/data-store 

Any data written to this file system is written to the Amazon EBS volume and is transparent to applications using the device.

An Amazon EBS volume can be detached from an instance by either explicitly detaching the volume or terminating the instance.

Note: A volume must be unmounted inside the instance before being detached. Failure to do so will result in damage to the file system or the data it contains.

In this example, the user unmounts the volume and explicitly detaches it from the instance.

# umount /mnt/data-store $ ec2-detach-volume vol-4d826724 -i i-6058a509

ATTACHMENT vol-4d826724 i-6058a509 /dev/sdh detaching 2008-02-14T00:00:17+0000

In this example, the user detaches from the volume by terminating the instance.

$ ec2-terminate-instances i-6058a509

INSTANCE i-6058a509 running shutting-down

To verify the volume is detached from the instance, the user calls the DescribeVolumes function.

$ ec2-describe-volumes

VOLUME vol-4d826724 800 in-use 2008-02-14T00:00:00+0000

ATTACHMENT vol-4d826724 i-6058a509 /dev/sdh detaching 2008-02-14T00:00:17+0000

VOLUME vol-50957039 1 available 2008-02-091T00:00:00+0000

VOLUME vol-6682670f 1 in-use 2008-02-11T12:00:00+0000

ATTACHMENT vol-6682670f i-69a54000 /dev/sdh attached 2008-02-11T13:56:00+0000

The preceding example shows the volume vol-4d826724 detaching from instance i-6058a509. After a short while, the volume will be available again as shown in the following example.

$ ec2-describe-volumes vol-4d826724

VOLUME vol-4d826724 800 available 2008-02-14T00:00:00+0000

VOLUME vol-50957039 1 available 2008-02-091T00:00:00+0000

VOLUME vol-6682670f 1 in-use 2008-02-11T12:00:00+0000

ATTACHMENT vol-6682670f i-69a54000 /dev/sdh attached 2008-02-11T13:56:00+0000

After writing data to the Amazon EBS volume, you can periodically create a snapshot of the volume which can be used as a baseline for new volumes or for data backup.In this example, the user creates a snapshot of the vol-4d826724 Amazon EBS volume using the CreateSnapshot function:

$ ec2-create-snapshot vol-4d826724

SNAPSHOT snap-78a54011 vol-4d826724 pending 2008-02-15T09:03:58+0000

As the data is written to S3, the snapshot completes asynchronously and the volume's status indicates "pending."Describing a SnapshotThe user views the list of snapshots (including their status) using the DescribeSnapshots function.

$ ec2-describe-snapshots snap-78a54011

SNAPSHOT snap-78a54011 vol-4d826724 pending 2008-02-15T09:03:58+0000 60%

The snapshot for the volume vol-4d826724 was initiated at 2008-02-15T09:03:58+0000, is processing, and is 60% complete). When the snapshot is complete, the status will change to "completed."

$ ec2-describe-snapshots snap-78a54011

SNAPSHOT snap-78a54011 vol-4d826724 completed 2008-02-15T09:03:58+0000 100%

The user creates the new volume from the "snap-78a54011 snapshot.

$ ec2-create-volume --snapshot snap-78a54011 -z us-east-1a

VOLUME vol-4282672b 800 creating 2008-02-15T09:15:11+0000

This volume can be attached to any instance.

The user can also elect to increase the size of the volume being created from the snapshot. This will present a larger block device presented to an instance once attached. Tools appropriate to the file system on the device, will have to be used to resize the file system in order to take advantage of the increased capacity. Example to create a 900 GiB volume:

$ ec2-create-volume —s 900 --snapshot snap-78a54011 -z us-east-1a

VOLUME vol-4582672c 900

After a volume is no longer needed, it can be deleted. Once deleted, its data is deleted and it cannot be attached to any instance.

$ ec2-delete-volume vol-4282672b

VOLUME vol-4282672b

Note: Deleting a volume will not affect any snapshot that was taken from the volume. Snapshots taken from this volume can be used to recreate the volume at any time in the future.Delete a SnapshotAfter a snapshot is no longer needed, it can be deleted.In this example, the user deletes an old snapshot snap-78a54011 using the DeleteSnapshot function.

$ ec2-delete-snapshot snap-78a54011 SNAPSHOT snap-78a54011

Note: The DeleteSnapshot function does not affect any volume(s) from which the snapshot was created. Data on these volume(s) continue to persist and these volume(s) continue in their current state.

• Can I DeleteVolume while attached to instance?
  • No, the volume needs to be detached first.
• Can I attach multiple volumes to one instance?
  • Yes
• Can I attach a volume to more than one instance at a time?
  • No, a volume can only be attached to one instance at a time.
• What happens to the volume if the instance is terminated?
  • The volume becomes detached.  It will continue to run and is available for attaching to an instance until the user deletes it.
• What happens to the volume if the instance is failing?
  • The volume must be explicitly detached, or the instance must be terminated.
• Will Snapshots of a volume be deleted when I delete a volume?
  • No, they must be explicitly deleted using the DeleteSnapshot API.
• Is there a limit to the number of volumes or snapshots that I can create?
  • Yes, the default limit is 20.
• Can a volume be attached to any instance?
  • A volume can be attached to any instance of yours that is running within the same availability zone.