Building an AWS CloudFormation custom resource to manage Amazon RDS point-in-time recovery

Amazon RDS makes it easy to set up, operate, and scale a relational database in the cloud. It provides cost-efficient and resizable capacity while automating time-consuming administration tasks such as hardware provisioning, database setup, patching, and backups. It frees you to focus on your business logic and application features, leaving the heavy lifting to AWS.

With the AWS shared responsibility model, you are responsible for safeguarding data and ensuring proper backups and recovery in case of a disaster. A backup is a snapshot of your dataset at a given point of time. Design a recovery strategy to restore all the changes that happened after the last full backup.

Using RDS, you can restore a database instance to a specific point in time, creating a new database instance. You can perform the point-in-time recovery using the AWS Management Console, the AWS CLI, or the RDS API.

There are individual cases in which organizations restrict the console and AWS CLI access to the database administrator. Currently, AWS CloudFormation doesn’t support point-in-time recovery, so there is an AWS CloudFormation custom resource as a workaround. Custom resources enable you to write custom provisioning logic in templates that AWS CloudFormation runs anytime you create, update (if you changed the custom resource), or delete stacks. For example, you might want to include resources that aren’t available as AWS CloudFormation resource types. You can include those resources by using custom resources. That way, you can still manage all your related resources in a single stack.

This post discusses how to perform the point-in-time recovery for an RDS database to any time in the past during the backup retention period using the custom resource AWS Lambda function.

Solution overview

The RDS service stores all the transaction logs for database instances in Amazon S3 every 5 minutes. In the console, you can see this property as the latest restore time for the database instance. You can restore to any point in time during your backup retention period.

This post goes through the following steps:

1. Creating a Lambda function and associated IAM role using the provided AWS CloudFormation template.
2. Creating another AWS CloudFormation stack to invoke the Lambda function with the required parameters and validate the recovery of the database to the time specified.

Prerequisites

To perform this solution, first complete the following:

• Create and activate an AWS account.
• Have an existing RDS database with sample schemas and data.
• Enable automated backups on the database to allow the restoration up to the last 5 minutes.

The AWS CloudFormation templates and the sample code that this post provides both use hardcoded information. These are for evaluation purposes only. Don’t use these for production without thorough testing.

Recovery steps

To deploy a Lambda function using AWS CloudFormation and test the solution, complete the following steps.

Creating the Lambda function

First, create a custom resource Lambda function that can restore the database to a time in the past. This function uses the RDS snapshots to achieve point-in-time recovery. You can use this downloadable template to set up the resources.

1. To launch directly through the console, choose Launch Stack.
   
2. Choose Next.
3. On the Specify stack details page, for Stack, enter a unique name, for example pitr-blog-custom-resource.
4. Choose Next
5. On the Configure stack options page, for Tags, specify any optional tags.
6. Choose Next.

7. On the Review page, select I acknowledge that AWS CloudFormation might create IAM resources.
8. Choose Create Stack.

It can take up to 5 minutes for the provisioning to complete. When the stack status shows CREATE_COMPLETE, review the created outputs and the resources by navigating to their respective tabs.

Testing the solution

This step requires the exported Lambda function name from the previous step, so ensure that the stack creation is complete and the ARN appears in the outputs. You can use this downloadable template to set up the resources.

1. To launch this stack directly through the console, choose Launch Stack.
   
2. Choose Next.
3. On the Specify stack details page, some of the parameters are predefined for ease of orchestration. For the rest, you must provide the values.

This post depicts point-in-time recovery solution using two scenarios. The first is performing a point-in-time recovery to create a new database based on the particular timestamp.

1. Enter the parameters with the appropriate values.
   • For pRenameExistingDBInstance, choose No. This action restores the point-in-time recovery to a new database, which is ideal for cases when you must restore a subset of your database.
2. After you specify the template details, choose Next.
3. When entering a restore time, ensure that the RDS database you are trying to restore has the required snapshot backups present for that period of time.

4. On the Configure Stack Options page, specify optional Tags and choose Next.
5. On the Review page, choose Create stack.
6. Navigate to the RDS console.

It takes up to few minutes to see a new database instance creation in progress. This new database is a point-in-time copy of the database you entered at the time of creating this stack.

The second scenario is performing a point-in-time recovery to an existing database based on the particular timestamp.

1. Enter the parameters with the appropriate values.
   • For pRenameExistingDBInstance, choose Yes. This action restores the point-in-time recovery to the same existing database and is ideal for restoring an existing business critical database in cases when something wrong with the production database.

2. After you specify the template details, choose Next.

When entering a restore time, ensure that the RDS database you are trying to restore has the required snapshot backups present for that time period.

3. On the Configure Stack Options page, specify optional Tags and choose Next.
4. On the Review page, choose Create stack.
5. Navigate to RDS console.

It takes up to few minutes to see the renamed existing instance and the new database instance from the point-in-time recovery.

Conclusion

This post showed the process of performing a point-in-time recovery to any time in the past during the backup retention period for RDS database engines using AWS CloudFormation. This solution provides database administrators an additional way to recover databases using an AWS CloudFormation custom resource and easily integrate with their existing recovery processes.

About the Authors

Udayasimha Theepireddy (Uday) has worked as a Sr. Cloud Database Architect at Amazon Web Services since November 2017. He works with internal Amazon customers to move several services from on-prem Oracle to Aurora, RDS PostgreSQL, RDS MySQL and Redshift databases.

Satya Vajrapu is a DevOps Consultant with Amazon Web Services. He works with AWS customers to help design and code modules for infrastructure, application and processes.