Amazon RDS Proxy is a fully managed, highly available database proxy for Amazon Relational Database Service (Amazon RDS) that makes applications more scalable, more resilient to database failures, and more secure.
Many applications, including those built on modern serverless architectures using AWS Lambda, can have a large number of open connections to the database server, and may open and close database connections at a high rate, exhausting database memory and compute resources. Amazon RDS Proxy allows applications to pool and share connections established with the database, improving database efficiency and application scalability. With RDS Proxy, failover times for Aurora and RDS databases are reduced by up to 66% and database credentials, authentication, and access can be managed through integration with AWS Secrets Manager and AWS Identity and Access Management (IAM).
Amazon RDS Proxy can be enabled for most applications with no code change, and you don’t need to provision or manage any additional infrastructure. Pricing is simple and predictable: you pay per vCPU of the database instance for which the proxy is enabled. This tutorial uses Amazon RDS with MySQL compatibility, but you can follow a similar process for other database engines supported by Amazon RDS Proxy. These concepts can also be applied to serverless architecture with Amazon RDS.
In this tutorial, you learn how to create an Amazon RDS Proxy and connect it to an existing Amazon RDS MySQL Database. The same process will apply to PostgreSQL as well. You use the MySQL/PSQL client on an Amazon EC2 instance to make a connection to the RDS MySQL/PostgreSQL Database through the RDS Proxy. When there are differences between the two engines, such as database endpoints and clients, we have provided detailed instructions.
Prerequisites
Amazon RDS Proxy requires that you to have a set of networking resources in place, such as:
- an Amazon Virtual Private Cloud (Amazon VPC),
- two or more subnets across different Availability Zones,
- an Amazon RDS database and Amazon EC2 instances within the same VPC, and
- an Internet gateway.
If you've successfully connected to existing RDS MySQL database instances, you already have the required network resources set up. This tutorial requires that your account is set up with an EC2 instance and an RDS MySQL instance in the same VPC. If you do not have these instances set up, then you can follow the RDS and EC2 instructions to provision the instances in the default VPC.
| About this Tutorial | |
|---|---|
| Time | 10 minutes |
| Cost | Less than $1 |
| Use Case | Databases |
| Products | Amazon RDS, AWS Secrets Manager |
| Audience | Database administrators, Developers |
| Level | Intermediate |
| Last Updated | August 7, 2020 |
Step 1: Verify security groups and database connectivity
For this step, you verify the inbound and outbound rules of your security groups, then verify connectivity from a current EC2 instance to an existing RDS database instance.
1.1 — Open the Amazon VPC dashboard and sign in with your AWS account credentials. If you do not have an AWS account, create a new AWS account to get started.
Already have an account? Log in to your account
1.2 — Choose the Region drop-down and select the AWS Region where your existing RDS and EC2 instances are located. This tutorial uses the US East (Ohio) Region.
1.3 — In the left navigation pane, choose Security Groups.
This tutorial uses two VPC security groups:
- EC2-sg: This security group is attached to the EC2 instance and allows only SSH connection inbound to the EC2 instance and any outbound connectivity.
- RDS-sg: This security group is attached to the RDS instance and allows only TCP connection on port 3306 from the EC2 instance (shown as EC2-sg) and any outbound connectivity.
1.4 – Select the EC2-sg.
- Choose the Inbound Rules tab and verify that the type is SSH.
- Choose the Outbound Rules tab and verify that the Type is All traffic.
1.5 — Select the RDS-sg.
- Choose the Inbound Rules tab and verify the Source is the EC2-sg.
- Choose the Outbound Rules tab and verify the Type is All traffic.
1.6 — Navigate to the RDS console, choose Databases, then choose your existing RDS MySQL DB instance. On the Connectivity & security tab, make a note of the instance Endpoint.
1.7 — Navigate to the EC2 console, choose Running instances, then choose the EC2 instance from which you want to test connectivity to the RDS DB instance. Choose Connect.
1.8 — In the Connect to your instance dialog box, choose EC2 Instance Connect (browser-based SSH connection). A browser window opens displaying the EC2 instance command line interface (CLI).
1.9 — In the EC2 instance CLI, test the connectivity to the RDS DB instance using the following command:
mysql -h <RDS DB endpoint> -P 3306 -u <username> -pWhen prompted, type your password and press Enter.
The CLI returns a message showing that you have successfully connected to the RDS DB instance. If you are unable to connect from the EC2 instance to the RDS instance, verify that both of the instances are in the same VPC and that the security groups are set up correctly.
Step 2: Store database credentials in AWS Secrets Manager
For this step, you store your database credentials in AWS Secrets Manager. Each database user account that the proxy accesses requires a corresponding secret in AWS Secrets Manager. Amazon RDS Proxy uses these secrets to maintain a connection pool to your database.
2.1 — Navigate to the Secrets Manager section of your AWS Management Console and choose Store a new secret.
2.2 – In the Select secret type box, choose Credentials for RDS database. Then, type the user name and password that you used when creating your database.
2.3 – Select the DefaultEncryptionKey and then choose the corresponding RDS database for the secret to access. Choose Next.
2.4 – In the Secret name and description section, give your secret a name and description so that you can easily find it later. Then, choose Next.
2.5 — AWS Secrets Manager allows you to configure automatic secret rotation for your secrets. This is a smart, easy way to enhance the security of your application. For more information, see Rotating Your AWS Secrets Manager Secrets.
Setting up secret rotation is outside the scope of this tutorial, so choose the Disable automatic rotation option, and then choose Next.
2.6 — The Secrets Manager console shows you the configuration settings for your secret and some sample code that demonstrates how to use your secret. Scroll to the bottom of the page and choose Store to save your secret.
2.7 — After creating the secret, the Secrets Manager page displays your created secrets. Choose your tutorial-secret.
In the Secret details box, it displays the ARN of your secret. Copy this value, as you need it later in this tutorial.
Step 3: Create IAM role and policy
In this step, you create the AWS Identity and Access Management (IAM) role and policy that allows RDS Proxy access to the secrets you created in AWS Secrets Manager.
3.1 — Navigate to IAM dashboard in the AWS Management Console. In the navigation pane of the IAM dashboard choose Roles, then Create Role.
3.2 — For Select type of trusted entity, choose AWS service. For Choose a use case, select RDS.
3.3. For Select your use case, choose RDS - Add Role to Database, and choose Next: Permissions.
3.4 – Choose Create policy and select the JSON tab. Delete the existing policy statements.
3.5 – Add the following new policy statement, substituting your secret ARN value for the example listed below. Then, choose Review policy.
(This policy statement is described in Setting Up AWS Identity and Access Management (IAM) Policies in the Amazon RDS User Guide.)
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"secretsmanager:GetRandomPassword",
"secretsmanager:CreateSecret",
"secretsmanager:ListSecrets"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "secretsmanager:*",
"Resource": [
"your_secret_ARN"
]
}
]
}
3.6 — In the Review policy section, give your policy a name and description so that you can easily find it later. Then, choose Create policy.
3.7 — Choose Roles and then choose Refresh.
3.8 — In the Search box, type tutorial and select the tutorial-policy. Choose Next: Tags.
3.9 — Skip the tagging section and choose Next: Review.
3.10 — In the Review section, give your role a name and description so that you can easily find it later. Then, choose Create role.
3.11 — Confirm that the role is created.
Step 4: Create an RDS Proxy
In this step, you create an RDS Proxy and configure the proxy for the security group you verified in Step 1, the secret you created in Step 2, and the role you created in Step 3.
4.1 — Navigate to the RDS console. In the RDS navigation pane, choose Proxies, then Create proxy.
4.2 — In the Proxy configuration section, do the following:
- For Proxy identifier, give the proxy an easily identifiable name.
- Clear the Require Transport Layer Security check box.
- Leave the Idle client connection timeout as the default 30 minutes.
4.3 — In the Target group configuration section, for Database, choose the RDS MySQL DB instance to be associated with this RDS Proxy. (This RDS DB instance is the same instance you verified connectivity to in Step 1.) For Connection pool maximum connections, keep the default value of 100.
4.4 — In the Connectivity section, do the following:
- For Secrets Manager secret, choose the secret you created in Step 2.
- For IAM role, choose the role you created in Step 3.
- For IAM Authentication, keep the default setting of Disabled. (This tutorial uses DB credentials to connect with the RDS Proxy so IAM Authentication is not used.)
- For Subnets, choose a minimum of two subnets in different Availability Zones.
- Expand Additional connectivity configuration, and for VPC security group, choose the existing RDS-sg security group.
4.5 — In the Advanced Configuration section, keep the default selection for Enhanced logging. Select the service agreement check box and choose Create proxy.
4.6 — Wait for the proxy status to change from Creating to Available, then select the proxy.
4.7 — In the Proxy configurations section, make a note of the Proxy endpoint and confirm all other parameters are correct.
Step 5: Connect to RDS DB through RDS Proxy
In this step, you connect to the RDS DB instance from your EC2 instance. However, instead of connecting directly, the EC2 instance connects to the RDS DB instance through your RDS Proxy.
5.1 — Navigate to the EC2 console. In the EC2 navigation pane, choose Running instances, then select the EC2 instance that you tested connectivity from in Step 1. Choose Connect.
5.2 — In the Connect to your instance dialog box, choose EC2 Instance Connect (browser-based SSH connection), and then choose Connect.
5.3 — In the EC2 instance CLI, use the following command to connect to the RDS instance through the RDS Proxy endpoint:
mysql -h <proxy endpoint> -P 3306 -u <username> -pWhen prompted, type your password and press Enter.
The CLI returns a message showing that you have successfully connected to the RDS DB instance via the RDS Proxy endpoint.
Step 6: Monitor your RDS Proxy connection
In this step, you use Amazon CloudWatch to monitor proxy metrics, such as client and database connections.
6.1 — Navigate to the CloudWatch console. In the CloudWatch navigation pane, choose Metrics, then choose RDS, Per-Proxy Metrics.
6.2 — In the Search box, type the name of your proxy.
6.3 — In the metrics list, choose ClientConnections and DatabaseConnections. For the display option, choose Number.
The ClientConnections metric shows the current number of client connections to the RDS Proxy reported every minute. The DatabaseConnections metric shows the current number of database connections from the RDS Proxy reported every minute. For details on all metrics, see Monitoring RDS Proxy.
The resulting graph shows that there is one client connection (EC2 to RDS Proxy) and one database connection (RDS Proxy to RDS DB instance). This data confirms the connection you made in Step 5.
Step 7: Clean up
In the following steps, you clean up the resources you created in this tutorial.
Delete RDS Proxy
7.1 — Navigate to the RDS console, and in the left pane, choose Proxies.
7.2 — Choose the tutorial proxy.
7.3 — Choose Actions, then choose Delete.
7.4 — In the dialog box, type delete me and choose Delete.
The status of the proxy changes to Deleting. When complete, the proxy is removed from the list.
Delete secret
7.5 — Navigate to the Secrets Manager console.
7.6 — Choose the tutorial secret.
7.7 — Choose Actions, then choose Delete secret.
7.8 — For safety, Secrets Manager requires a waiting period before a secret is permanently deleted. Use the default period of 30 days and choose Schedule deletion.
Delete IAM role and policy
7.9 — Navigate to the IAM console, and in the navigation pane, choose Roles.
7.10 — Search for the tutorial-role and then select the check box next to the role.
7.11 — At the top of the page, choose Delete role.
7.12 — In the confirmation dialog box, choose Yes, Delete.
7.12 — In the IAM navigation pane, choose Policies.
7.13 — Search for the tutorial-policy and select the check box next to the policy.
7.14 — Choose Policy actions, and then choose Delete.
7.15 — Confirm that you want to delete the policy, and then choose Delete.
Delete other resources
If you created a new EC2 instance, new RDS instance, and corresponding security groups for this tutorial, delete those resources also.
Congratulations
You have created an Amazon RDS Proxy to pool and share database connections, monitored the proxy metrics, and verified the connection activity of the proxy.
Recommended next steps
Learn more about Amazon RDS features
Find out more about the features of Amazon RDS with the Amazon RDS User Guide.
Best practices with Amazon RDS
Learn about general best practices and options for working with Amazon RDS.
Learn more about Amazon RDS Proxy
If you want to learn more, read the Using Amazon RDS Proxy with AWS Lambda blog post and see Managing Connections with Amazon RDS Proxy.