Automating Cross-Region and Cross-Account Snapshot Copies with the Snapshot Tool for Amazon Aurora

This post covers how to install the Snapshot Tool for Amazon Aurora to automate copying snapshots across AWS accounts and Regions.

Although this is not a comprehensive list, here are a few situations where you might find this tool useful:

• To keep a copy of your data in a different account for security purposes.
• To keep a copy of your data in a different Region for disaster recovery.
• As an easy way to update development/testing environments with fresh production data when development/testing runs on a different account than production.

The Snapshot Tool for Aurora automates the task of creating manual snapshots—copying them into a different account and a different Region and deleting them after a specified number of days. The tool also allows you to specify the backup schedule (at what times and how often) and a retention period in days.

If you want to see these features as a part of the Amazon RDS service, please leave your feedback in the comments section.

This post assumes that you are familiar with Amazon RDS and Amazon Aurora backups. For information about managing snapshots, refer to the Amazon RDS documentation.

How the Snapshot Tool for Aurora works
The Snapshot Tool for Aurora uses AWS Lambda functions to create, copy, and delete tasks. These tasks are orchestrated using AWS Step Functions to make sure that each of them are retried, with the scheduling powered by Amazon CloudWatch Events. If any of these tasks ends in an error, it triggers a CloudWatch Alarm, which pushes a message to an Amazon SNS topic. You can subscribe to these SNS topics to receive an email, trigger a Lambda function of your own, post a message to a Slack channel, etc.

Snapshots that are created by this tool follow a simple naming convention: <clusterIdentifier-YYYY-MM-DD-HH-MM>. They also receive tags that identify them as being created by the <Snapshot Tool for Aurora>. The tool deletes old snapshots by looking at the time stamp in the name, but it only deletes those that are tagged correctly. This prevents the tool from deleting snapshots that you created manually.

If you decide to change the cluster identifier, it doesn’t affect your old snapshots (they keep their name and are deleted according to your setup). New snapshots are created with the new identifier.

Now that we’ve had a look at the architecture, let’s get started with setting it up on your accounts. This post discusses the setup for cross-region and cross-account copies, but the tool can easily accommodate other use cases (such as taking snapshots but not sharing, copying cross-account but not cross-region, disabling deletion of old snapshots, or a combination of those use cases). For more information, see the README.md file.

Deploying the Snapshot Tool for Aurora
The tool is deployed by running two AWS CloudFormation templates: one in the source account (namely, where your Aurora clusters are running), and the other in the destination account. You can find these templates in the cftemplates directory of the GitHub repository as snapshot_tool_aurora_source.json and snapshot_tool_aurora_dest.json.

Source account setup
The following instructions must be executed on the source account and in the same Region where you are running your Aurora clusters. If AWS Step Functions is not available in that Region, you need to use the SourceRegionOverride. See the README.md file for instructions.

After you run the snapshot_tool_aurora_source.json template, you are presented with a screen similar to the following:

The BackupSchedule parameter sets the time at which your snapshots are created. It needs to be specified in the CloudWatch Events cron format (for more information, see Schedule Expressions for Rules). The default value is set to run every day at 1:00 A.M. UTC.

The BackupInterval parameter tells the tool how often you want your clusters to be backed up to snapshots. The default is every 24 hours. The Lambda function that is responsible for triggering the backups looks at the existing snapshots and the number of hours set in the BackupInterval parameter to decide whether a new snapshot needs to be created. For example, if you set BackupSchedule to “0 0,8,16 * * ? *” (as I did in the preceding example) to have them backed up at 0, 8, and 16 UTC, you also need to set BackupInterval to 8 hours. Following the same example, if a snapshot that is newer than 8 hours exists, the tool does not create a new snapshot at that time.

To share the snapshots, you need to set the account ID in the DestinationAccount parameter. If you don’t want to copy your snapshots to a different account, you can leave the value at its default (‘000000000000’). But make sure that you also set ShareSnapshots to FALSE, so you avoid creating unnecessary objects. If you decide later on that you want to copy snapshots across accounts, you can simply update the CloudFormation stack and set those parameters accordingly.

ClusterNamePattern allows you to filter the clusters that are backed up with this tool. It accepts a Python regex that is passed directly to the Lambda functions as environment variables. For example, you could set it to “prod” and all clusters that do not contain “prod” in their name are ignored. Bear in mind that this pattern applies to the cluster identifier, not the instance name. You can find the cluster identifiers in the Clusters tab of the Amazon RDS console (see the following image).

Finally, you need to set the number of days you’d like to keep snapshots before deleting them, using the RetentionDays parameter. You can set up different RetentionDays in the source and destination account depending on your specific use case. But you need to make sure that snapshots are not deleted before they are copied to the destination account. A few days is usually enough (the default is 7), but it depends on the amount of data and how often you create snapshots. So you need to experiment a bit. You can disable this functionality completely by setting DeleteOldSnapshots to FALSE.

Destination account setup
Now that you are all finished with the source account, you need to set up copying on the destination account. You can do that by running the aurora_snapshot_tool_dest.json template. As with the source account setup, it’s best if you run this AWS CloudFormation template in the same Region where you are running your Aurora clusters. If Step Functions is not available in that Region, you can use the SourceRegionOverride parameter. Refer to the README.md file for more information.

If you run the template from the console, you are presented with a screen similar to this one:

The setup in the destination account is very similar, with only a few different parameters.

DestinationRegion needs to be set to the Region where you want your snapshots to be copied. Since copying a snapshot across accounts and across Regions must be done in separate steps, the tool first copies the snapshot locally (in-region). Once that copy is finished, the tool copies your snapshots to the destination Region and deletes the extra copy at the end. Copying snapshots across accounts and Regions can take several hours, so all these actions are achieved by multiple invocations of the same Lambda function using AWS Step Functions.

SnapshotPattern, similarly to ClusterNamePattern in the source account, allows you to specify which snapshots should be copied to the destination Region. For example, you could set ClusterNamePattern in the source account to the default ALL_CLUSTERS in order to take snapshots of all your Aurora clusters. Then set SnapshotPattern to “prod” in the destination account to copy snapshots only from your production clusters.

Encrypted snapshots
If you are using encryption at rest on your Aurora clusters, you need to perform a few more steps before you are finished. If you’re not using encryption at rest, you can safely skip this section. The following setup involves sharing your master encryption key from the source to the destination account.

For more information about sharing keys across accounts, you can refer to this AWS Support post.

Important: Make sure that you are using a custom encryption key (not the default RDS key) to encrypt your Aurora databases. You can verify this setting from the console by selecting one of your Aurora instances and choosing See Details.

IMPORTANT: Sharing snapshots across accounts with the default Amazon RDS KMS (Key Management Service) key is not supported and doesn’t work.

Source account setup
You need to sign in to the source account and provide access to the KMS key to the destination account. You can do this from the IAM console by choosing the Encryption keys section and then choosing your key.

After you select your key, scroll down to see a section that allows you to add Key Users. You should add the destination account there.

You might need to do this multiple times if you are using more than one key.

Destination account setup
Now that you’ve given your destination account permission to decrypt using your key in the source account, you need to create two keys in the destination account: one in the source Region (where your Aurora clusters run) and another one in the destination Region. KMS keys are available only in the Region where they are created. So you need to select the correct Region before choosing Create key.

Follow the steps in the IAM console to create both keys, but you can leave the permissions section empty. When you’re finished, copy the Amazon Resource Name (ARN) for both keys and put them in the KmsKeySource and KmsKeyDestination parameters of the AWS CloudFormation template. You can find the Key ARN in the details page for each key, like the following example:

Final verification
If you followed the preceding steps, you should see snapshots being created and copied across accounts and Regions. The easiest way to verify it is by looking at the Snapshots section of the Amazon RDS console. My console looks like the following after a few days of letting the tool run:

The Amazon RDS console on the destination account should look very similar.

You can subscribe to the SNS topics that are created by the AWS CloudFormation template to receive notifications of failures at any point in the process of taking and copying snapshots to their destination Region and account. You can find them in the Output section of the AWS CloudFormation console. For more information about Amazon SNS and how to configure notifications, see the Amazon SNS documentation.

I hope you find this tool useful. Please leave your feedback in the comments section!

About the Author

Marcelo Coronel is a technical account manager at Amazon Web Services. He works with Enterprise Support customers, and provides technical guidance and assistance to help them make the best use of the AWS platform. In his spare time, he loves travelling and is always thinking of new places to visit.