AWS Database Blog

Use Amazon RDS Proxy and AWS PrivateLink to access Amazon RDS databases across AWS Organizations at American Family Insurance Group

The American Family Insurance Group of companies includes American Family Insurance, CONNECT (powered by American Family Insurance), The General, Homesite, and Main Street America Insurance. It is the nation’s twelfth-largest property and casualty insurance group, ranking number 301 on the Fortune 500 list. Across these companies, the group has nearly 13,000 employees nationwide.

The group required an enterprise-level solution to privately share centralized information for their common systems of record. This information was stored in Amazon Relational Database Service (Amazon RDS) and needed to be accessible to multiple lines of business. The information is used by commercial systems across the organization to drive process automation, aggregate clickstream data for marketing campaigns, and fulfill analytical or operational reporting needs.

To follow best practices, the company decided to isolate workloads in their own AWS account and network boundary. However, they still needed to grant access to the information across different AWS accounts and VPCs that had overlapping CIDR ranges.

In this post, we show how the organization created a solution for highly available, centralized database access. They achieved this by combining Amazon RDS Proxy with AWS PrivateLink, which allows for secure and private access to RDS databases. This solution enables clients from different AWS Organizations and accounts to connect to the database over PrivateLink, even if they have overlapping CIDR ranges.

Shared Amazon RDS connectivity

In this post, we refer to “consumer” and “database” accounts to demonstrate connectivity. The database account includes an RDS database (or Amazon Aurora cluster), RDS proxy, and PrivateLink endpoint service. The consumer account includes an interface VPC endpoint using the database account PrivateLink endpoint service.

Customers with multiple organizations, overlapping CIDR ranges across VPCs, or a centralized database management strategy may require shared access to a common RDS DB instance or Aurora cluster. These situations can be the result of merger and acquisition or by design, offering centrally defined entry points to specific database resources.

With overlapping CIDR ranges, you can use PrivateLink to create an endpoint service in the database account that will be accessible by AWS principals, such as other AWS accounts. A PrivateLink endpoint service is backed by a Network Load Balancer (NLB) with a target group. One option is to create a target group with the target type of IP addresses pointing to the RDS instance or Aurora cluster IP, as detailed in the related post Access Amazon RDS across VPCs using AWS PrivateLink and Network Load Balancer. However, IP addresses may change as a result of multiple Availability Zone failover or database maintenance activity. When using a supported DB engine, RDS Proxy creates a connection pool and automatically connects to the new underlying RDS DB instance or Aurora cluster instance in the event of a failover. Creating an RDS Proxy endpoint in multiple Availability Zones provides static IP addresses for the life of the endpoint, which can be set as the IP address target for an NLB.

After a PrivateLink endpoint is created in the database account, the allowed principals create an interface VPC endpoint in consumer accounts, which requires private connectivity to the RDS DB instance or Aurora cluster.

If connectivity to an RDS database is across AWS accounts, but the accounts reside within VPCs without overlapping CIDR ranges connected via AWS Transit Gateway, then refer to Use Amazon RDS Proxy to provide access to RDS databases across AWS accounts.

Let’s dive deeper into the specific solution the American Family Insurance Group deployed enabling secure Amazon RDS connectivity across organizations with overlapping VPC CIDR ranges.

Solution overview

We use two AWS accounts, each within a different organization. Each account uses their own VPC with overlapping CIDR ranges.

The following diagram illustrates the solution architecture.

Diagram illustrating the solution architecture.

Prerequisites

Before getting started, the following prerequisites must be met:

  1. Choose two accounts in separate organizations. This step is optional; standalone accounts will also work.
  2. In the database account, create a VPC with a minimum of two private subnets placed in different Availability Zones.
  3. Note the Availability Zone ID (for example, use1-az1) for the selected Availability Zone. This can be found on the subnet details tab on the Amazon Virtual Private Cloud (Amazon VPC) console.
  4. In the consumer account, create a VPC with a minimum of two private subnets using the same Availability Zone IDs from the DB account.
  5. Make sure the Amazon RDS or Aurora Region, engine, and version you’ll create are supported by RDS Proxy.

Create an RDS DB instance and RDS proxy in the database account

For this post, we use Amazon RDS for MySQL. To create an RDS for MySQL DB instance, complete the following steps:

  1. On the Amazon RDS console, choose DB instances in the navigation pane.
  2. Choose Create database.
  3. Choose Amazon RDS for MySQL.
  4. For Version, choose the latest available version.
  5. For Templates, choose production template.
  6. For Availability and durability, select Multi-AZ DB instance.
  7. In the Settings section, choose a DB instance identifier, set a user name and password, and confirm the password.
  8. For Instance configuration, choose a DB instance class.
  9. For Connectivity, choose the VPC created as a prerequisite.
  10. Choose Create an RDS Proxy.
  11. Create a new DB subnet group.
  12. For Public access, choose No.
  13. For VPC security group, create a new security group.
  14. Modify the custom rules in the security group for your DB instances to allow inbound connections on your database port (3306 for MySQL) from the subnet CIDRs you will use when creating the RDS proxy in the next step.
  15. For additional configurations, enter an initial DB name.
  16. Leave the other settings as default.

The following screenshot shows the configuration of an RDS for MySQL DB instance using the db.t3.micro instance class. Ensure that the status is available before moving to the next step; this typically takes a few minutes.

Screenshot shows the configuration of an RDS for MySQL DB instance using the db.t3.micro instance class

Identify the IP addresses assigned to the RDS Proxy endpoint

Next, we identify the IP addresses assigned to the VPC endpoint automatically created for the RDS Proxy endpoint.

  1. On the Amazon RDS console, choose Proxies in the navigation pane.
  2. Choose the proxy identifier name created in the previous step.
  3. Locate the proxy endpoint you wish to share and copy the endpoint DNS name.
  4. Using a DNS lookup tool, such as Dig, look up the VPC endpoint ID using the RDS Proxy endpoint DNS name.
  5. Copy the VPC endpoint ID from the DNS lookup results.
    The VPC ID is identified by a leading vpce- followed by a series of characters up to but excluding the following dash (-), as shown in the following screenshot.

    Screenshot showing how to identify the VPC ID

  6. On the Amazon VPC console, choose Endpoints in the navigation pane.
  7. Locate the VPC endpoint ID copied from the DNS lookup results.
  8. Choose the Subnets tab and note the IP addresses for each subnet.

You will use the IP addresses when creating the PrivateLink endpoint service in the next step.

Create a PrivateLink endpoint service in the database account

Next, we complete three steps to create a PrivateLink endpoint service using the RDS proxy IP addresses as targets.

Create a target group in the database account

Complete the following steps to create a target group using Amazon Elastic Compute Cloud (Amazon EC2):

  1. On the Amazon EC2 console, choose Target groups in the navigation pane.
  2. Choose Create target group.
  3. For Basic configuration, select IP addresses.
  4. Set a target group name.
  5. Change Protocol to TCP and change Port to your RDS instance port (the MySQL default is 3306).
  6. Select the VPC used previously for RDS Proxy.
  7. For Register targets IP addresses, specify the target entries for each of the IP addresses noted when identifying the IP addresses assigned to the RDS Proxy endpoint in the previous step.

The target health status will change to Healthy when the configured health checks are complete.

  1. Choose Create target group.

Ensure all targets are healthy, as shown in the following screenshot of the target group.

Screenshot showing all targets are healthy

Create a Network Load Balancer in the database account

Complete the following steps to create a Network Load Balancer:

  1. On the Amazon EC2 console, choose Load balancers in the navigation pane.
  2. Choose Create load balancer.
  3. Under Network Load Balancer, choose Create.
  4. For Basic configuration, provide an NLB name.
  5. Set the scheme to Internal.
  6. For Network mapping, choose the VPC used previously for RDS Proxy.
  7. Choose each subnet used previously for RDS Proxy.
  8. For Listeners and routing, change Protocol to TCP and change Port to your RDS instance port (the MySQL default is 3306).
  9. For Default action, choose the target group created in the previous step.

Create a VPC endpoint service in the database account

Complete the following steps to create a VPC endpoint in the database account:

  1. On the Amazon VPC console, choose Endpoint services in the navigation pane.
  2. Choose Create endpoint service.
  3. In the Endpoint service settings section, provide a name.
  4. For Available load balancers, choose the NLB created in the previous step.
  5. Choose Create.
  6. On the Allow principals tab, choose Allow principals.
  7. Enter the ARN of your consumer account in the format arn:aws::<aws-account-id>:root.
    This will allow any user or role in the consumer account to use the PrivateLink. The principal can be further restricted to a specific user or role.
  8. Under Details for the Endpoint Service, copy the service name and save it for later.

Accepting the consumer account endpoint connection will occur in the next step.

At this point, we’ve created an RDS DB instance with an RDS proxy along with a VPC PrivateLink endpoint service. Next, we log in to the consumer account and create a VPC endpoint to consume the VPC endpoint service created in the database account.

Create a VPC endpoint in the consumer account

Complete the following steps to create a VPC endpoint in the consumer account:

  1. Log in to the consumer account.
  2. On the Amazon VPC console, choose Endpoints in the navigation pane.
  3. Choose Create endpoint.
  4. For Endpoint settings, provide a name.
  5. For Service category, select Other endpoint services.
  6. Enter the endpoint service name copied from the previous step and verify the service.
  7. For VPC, choose a VPC with at least two subnets.
  8. For Subnets, choose at least two subnets.
  9. For Security group, choose a security group allowing TCP to your RDS instance port (the MySQL default is 3306).
  10. Choose Create endpoint.
  11. In the database account, on the VPC endpoint services Endpoint connections tab, select the new endpoint connection request.
  12. On the Actions menu, choose Accept endpoint connection request.
  13. Enter accept to confirm, then choose Accept.

After a few minutes, the status will change from Pending to Available.

Test the connection

Now we can test the connection. Using a client (such as an EC2 instance with the MySQL client for Linux installed) in the consumer account, we can use the VPC endpoint DNS as the host of our RDS instance or cluster.

The following screenshot shows a successful MySQL client connection test using PrivateLink.

Screenshot shows a successful MySQL client connection test using PrivateLink.

Clean up

After you’ve finished, if the resources you created are no longer needed, delete the VPC endpoint service, Network Load Balancer, and the target group followed by the RDS proxy and RDS DB instance to prevent incurring additional charges.

Conclusion

In this post, we showed you how the American Family Insurance Group successfully implemented a highly available solution to share centralized information stored in RDS databases across different AWS accounts and VPCs with overlapping CIDR ranges. By combining RDS Proxy with PrivateLink, they were able to achieve secure and private access to their databases, enabling clients from different organizations to use system of record information for customer applications and analytics applications.

For more information about the American Family Insurance Group, see the amfam.com website. Leave your feedback in the comments section to further improve this post.

About the authors

Jarrid Kleinfelter is a Senior Solutions Architect at AWS.

Photo of author Charles TimmonsCharles Timmons is a Cloud Platform Engineer at the American Family Insurance Group.