Automate AWS Backups with AWS Service Catalog

If you’re an organization with multiple AWS accounts and independent teams, cloud governance can seem a daunting task. The complexities of balancing developer velocity with centralized governance risks can slow down the innovation you’re trying to speed up. Fortunately, AWS Service Catalog, and AWS Backup help to implement a well-architected approach to self-service while meeting your enterprise requirements.

AWS Service Catalog allows organizations to create and centrally manage approved catalogs of IT services to meet your compliance requirements. AWS Service Catalog does this while enabling your development team end users to deploy only the approved IT services they need. One solution for extending this is AWS Backup. AWS Backup is a fully managed backup service that allows you to centrally configure backup policies and monitor backup activity for AWS resources.

In this post, we demonstrate how to automate backups for AWS Service Catalog products using AWS Backup. With this solution, you can standardize resource provisioning with tags, and ensure compliance with backup policies. You can also identify specific resources for backup. In this self-service model, as your development teams provision resources they are proactively identifying resources aligned to the right backup policies.

AWS Service Catalog overview

AWS Service Catalog is based on products, which are the AWS CloudFormation templates used to provision resources in your AWS account. AWS Service Catalog administrators group products together with configuration information and share them as portfolios. AWS Service Catalog also includes templates for AWS best practices in the Getting Started Library. These reference architectures can be used to help you jumpstart provisioning with AWS Service Catalog.

An important practice for deploying and managing resources at scale is to use case-sensitive resource tags in accordance with an enterprise tagging strategy. AWS Service Catalog aligns to these best practices by providing a TagOptions library. The TagOptions library can be used to ensure that each provisioned product is consistently, proactively, and properly tagged.

In relation to setting your tags for AWS Backups, you create the following TagOptions entries: backup=daily, backup=weekly, and backup=monthly. These options will ensure that provisioned products inherit the AWS Backup policies on deployment.

AWS Backup overview

With AWS Backup, you define your backup plan, which is a policy that determines frequency and retention of backups in a specified backup vault. This policy is applied to the protected resources, such as DynamoDB tables or Amazon Elastic File System (Amazon EFS) file systems. The backups defined by that policy are stored in an encrypted backup vault for safe keeping during the retention period. Resource tags simplify your specification of protected resources. That is, you configure your backup plans one time, and in doing so also target specific tags for identifying the resources you will backup. Any resources matching the tag selection rule will now be automatically included in your backup plan.

Within each AWS account, administrators can define policies that govern backups and retention.

We will be creating account level policies that align with AWS Backup templates. These will create backup plans with daily, weekly, and monthly backups. The resource assignments for these backup policies are provisioned based on the presence of the following tags: ‘backup=daily’, ‘backup=weekly’, and ‘backup=monthly’. These policies will be created later in this post using AWS CloudFormation.

Solution implementation

To build the components of this solution, you will be working with AWS Backup, AWS CloudFormation, and AWS Service Catalog. You will deploy the backup policies and provision a product with tags from the TagOption library that align to your backup plan.

Step 1: AWS Backup configuration

To get started, let’s navigate to AWS Backup. These steps demonstrate the manual procedures to build three backup plans and populate the backup tag in the AWS Service Catalog TagOption library. AWS Backups does support AWS CloudFormation to automate the steps as well.

Log in to your AWS account.
Go to AWS Backup in the AWS Management Console.
Create a new backup plan. These steps reference a backup plan named “BackupPlanWithDailyBackups”.

The template configures several of the options for the backup rules. NOTE: These templates, will use a default backup vault and an AWS Key Management Service encryption key for that vault. If you require different encryption keys or access policies for different groups of backups, you can create multiple backup vaults.
The next step is to Assign resources to implement tag-based backup.

This blog post was based on backup plans named: BackupPlanWithDailyBackups, BackupPlanWithWeeklyBackups, and BackupPlanWithMonthlyBackups associated with tags: backup=daily, backup=weekly, and backup=monthly. You will want to repeat these steps with appropriate configurations to match the remaining two backup plans.

You have now created backup policies that will be automatically applied when you provision a resource that has tags matching backup=daily, backup=weekly, or backup=monthly. As your needs for retention and vault access change, you could add additional backup policies for workloads such as backup=HIPAA.

Step 2: Provision AWS CloudFormation stack for AWS Service Catalog

For this example, we’ll use a portfolio that contains reference architecture products for Amazon Elastic Compute Cloud (Amazon EC2). We obtained these from The Getting Started Library in the AWS Service Catalog console. Although we’ll be using an EC2 portfolio as our example, you can enable automatic backups for a number of different AWS services.

Log in to your AWS account.
Go to AWS CloudFormation in the AWS Management Console.
Use the following button to launch the AWS CloudFormation stack that creates an AWS Service Catalog portfolio for use with this example.

Create stack page – Choose Next.
Specify stack details – Choose Next. (you can modify the stack name or the linked roles if you choose)
Configure stack options page – Choose Next.
Review Portfolio page.

- Checkbox: I acknowledge that AWS CloudFormation might create IAM resources and choose Create stack.

Checkbox: I acknowledge that AWS CloudFormation might require the following capability: CAPABILITY_AUTO_EXPAND
Choose Create Stack.

- Wait until the Stack status changes to CREATE_COMPLETE.

You have now created a portfolio of products – Service Catalog EC2 reference architecture – in AWS Service Catalog.

Step 3: Service Catalog configuration

To configure each resource provisioned is correctly tagged for backups, we’ll create TagOptions in AWS Service Catalog. These will be specific tags for backups that occur daily, weekly, and monthly.

Log in to your AWS account with your AWS Service Catalog administrator role.
Go to AWS Service Catalog in the AWS Management Console.
Under Administration, select Portfolios, then select your newly created portfolio.
In the “Groups, roles and users” tab, ensure that your portfolio is shared to the end users you want to enable for self-service provisioning.
Navigate to the TagOptions tab
Select the TagOption you would like to associate with this portfolio and choose Associate TagOption from the Actions menu.

NOTE: Through the TagOption Library, you can add additional TagOptions for the backup tag. You can also create new tags to reflect the metadata hierarchy you would like to implement.

Step 4: Service Catalog product provisioning

Now, whenever a Service Catalog end user provisions a product from this portfolio, they must specify one of the three tag option values for the backup plans. Let’s switch to the perspective of a developer and provision an Amazon EC2 Linux instance from Service Catalog. Although we’ll be using EC2 as our example, you can enable automatic backups for a number of AWS services. You will see we have several products in this portfolio.

Log in to your AWS account as an AWS Service Catalog end user.
Go to AWS Service Catalog in the AWS Management Console.
From the “Products List” choose Amazon Elastic Compute Cloud (EC2) Linux and choose “Launch Product”
Launch page – enter a name for your provisioned EC2 Linux product, then choose Next.
Parameters page.

- Fill in the details for the General Configuration.
- Note: you need to determine the public IP address you will use to connect to the EC2 instance to populate the Remote Access CIDR Block value. This should be as specific as possible to avoid opening up SSH access too broadly.
- Click Next

On the TagOptions page, the values you associated to this portfolio for the backup key will be available as choices for this selection. Also, the selection of this choice is required before provisioning the product. For this example, Choose the daily value, then Choose Next.

Notification page –Choose the notification options you prefer for provisioned products, then Choose Next.
Review Page – unless you want to change any values entered previously, you can click on Launch to begin provisioning your product.

Step 5: Verify your backups

From these steps, you have deployed backup policies and provisioned a properly tagged EC2 instance that will be associated to those backup jobs. To verify your work

Log in to your AWS account.
Go to AWS Backup in the AWS Management Console, select Jobs on the navigation pane. You will see the backup job status for each of your provisioned products that have the associated backup tag.

Conclusion

In this post we demonstrated how to integrate AWS Backup with AWS Service Catalog to provision products that are aligned with your backup policies. Your developers are able to self-provision products from an approved portfolio and your backup policies are automatically applied. There is completed with assurance that your developers are compliant with your enterprise backup policies.

AWS Cloud Operations & Migrations Blog