AWS Storage Blog
Taking crash-consistent snapshots across multiple Amazon EBS volumes on an Amazon EC2 instance
Amazon Elastic Block Store (Amazon EBS) enables you to back up volumes at any time using EBS snapshots. Snapshots retain the data from all completed I/O operations, allowing you to restore the volume to its exact state at the moment before backup (referred to as crash-consistency).
Many of our customers use snapshots in their backup or disaster recovery plans. A common use of snapshots is to create a backup of a critical workload such as a large database or file system. In some cases, these backups span multiple EBS volumes.
Previously when you combined multiple EBS volumes to meet your application or performance needs (such as RAID), you followed a multi-step process. This process ensured that the backups of all EBS volumes were in sync relative to each other. You had to pause I/O or stop the instance to temporarily disable write access, create snapshots for each of your volumes, and then resume I/O. This added complexity to your backup process. Now, you can simplify your backup process with a one-click solution to take exact point-in-time backups across multiple EBS volumes attached to an EC2 instance.
In this post, I demonstrate how to use the new capability to create crash-consistent snapshots across all the EBS volumes attached to an EC2 instance. I show you how to do this from both the AWS Management Console and using the new CreateSnapshots API with the AWS Command Line Interface (AWS CLI). I also walk you through how to use this capability with Amazon Data Lifecycle Manager (Amazon DLM) to automate snapshot management.
Taking multi-volume crash-consistent snapshots of your instance from the AWS Management Console
Log in to the EC2 console and choose Elastic Block Store, Snapshots, Create Snapshot.
For Resource type, choose Instance and select the instance from which to create crash-consistent snapshots. For this post, take snapshots of all of the EBS volumes on this instance, including the root volume. To back up only your data volumes, set the Exclude root volume flag.
During creation, snapshots are managed together. If any single snapshot in the volume set fails, all other snapshots are moved to error status for the volume set. You can monitor the progress of your snapshots using CloudWatch Events. After the snapshot process is complete, CloudWatch emits an event with the status and all relevant snapshot details for the particular instance.
After it’s created, each snapshot is treated like an individual snapshot. You can perform all operations such as restore, delete, and cross-region/account copy, similar to a single volume snapshot. As with individual snapshots, you can also tag your multi-volume snapshots upon creation. I recommend that you tag your multiple volume snapshots to manage them collectively during restore, copy, or retention.
Typically, multi-volume, crash-consistent snapshots are restored as a set. Tagging your set with instance-id, name, or other relevant details helps you to identify the snapshots that are in a crash-consistent set. You can also set the Copy tags from volume flag to automatically copy tags from the source volume to corresponding snapshots. This helps you set snapshot metadata such as access policies, attachment information, and cost allocation to match the source volume.
You are now ready to view your snapshots. After receiving the CloudWatch Events notification, check the EBS dashboard and see that snapshots for all the volumes were created at the same point-in-time.
Taking multi-volume crash-consistent snapshots of your instance from the CreateSnapshots API
To use the AWS CLI, see Creating an Amazon EBS Snapshot.
Use Amazon DLM to automate the creation and retention of your snapshots
You have seen how simple it is to create crash-consistent snapshots across multiple EBS volumes. A typical pattern that I see is that customers want to take snapshots on a regular schedule. Use Amazon DLM to provide this capability for snapshots.
You can create lifecycle policies to automate the creation and deletion of multiple volume snapshots. Alternatively, you can also write AWS Lambda functions or custom scripts to automate the management of these snapshots.
You can create the policies in the AWS Management Console, from the AWS CLI or using the Amazon DLM APIs. For this post, use the console.
Amazon DLM is powered by AWS resource tags, so you must tag your EC2 instances in order to use Amazon DLM to schedule multi-volume, crash-consistent snapshots. The following screenshot shows instances all suitably tagged with type.
Create a policy to take snapshots every 24 hours for all the instances that are tagged with type=prod.
Choose the IAM role and other configurations required for the policy. By choosing the variable tags, you can opt to tag all snapshots automatically with the corresponding instance-id and backup timestamp values. This helps you associate snapshots back to the source instance. To learn more about Amazon DLM policy configuration, see Automating the Amazon EBS Snapshot Lifecycle.
After the policy has been executed, you see that the policy created the first set of snapshots. In the following screenshot, you can see that all snapshots are properly tagged with the corresponding instance-id, backup timestamp, and type values.
Things to know
Here are a few things to keep in mind when you use the new EBS CreateSnapshots API.
Pricing—There is no additional cost to use the multi-volume snapshot capability. You pay the usual storage costs for EBS snapshots.
Performance—You no longer have to stop your instance or freeze I/O to take point-in-time backups across multiple EBS volumes attached to an instance. However, creating multi-volume, crash-consistent snapshots of an instance results in a brief I/O suspension that can last from a few milliseconds to a few seconds. It depends on the size and number of volumes attached to your instance.
- If your application is latency-sensitive, I also recommend that you schedule your backups during off-peak hours. You can schedule automated snapshots early in the morning in the zone where your instance is located rather than immediately at midnight. Snapshot creation typically peaks at midnight.
Availability—Multi-volume snapshot capability is available in US East (N. Virginia), EU (Ireland), US West (N. California), Asia Pacific (Singapore), Asia Pacific (Tokyo), US West (Oregon), South America (Sao Paulo), Asia Pacific (Sydney), EU (Frankfurt), Asia Pacific (Seoul), Asia Pacific (Mumbai), US East (Ohio), Canada (Central), EU (London), EU (Paris), Asia Pacific (Osaka), and EU (Stockholm).
Conclusion
Maintaining regular backups of your EBS volumes helps you protect against events like the accidental deletion or unintentional modification of your data.
In this post, you learned about a one-click solution to take backups across multiple EBS volumes while ensuring that the data on each volume is in sync. I also discussed how to manage snapshot inventory using Amazon DLM.
If you have any questions regarding this post, start a new thread on the Amazon EC2 forum or contact AWS Support.