How to securely provide database credentials to Lambda functions by using AWS Secrets Manager
April 29, 2022: This post has been updated based on working backwards from a customer need to securely allow access and use of Amazon RDS database credentials from a AWS Lambda function.
In this blog post, we will show you how to use AWS Secrets Manager to secure your database credentials and send them to Lambda functions that will use them to connect and query the backend database service Amazon RDS—without hardcoding the secrets in code or passing them through environment variables. This approach will help you secure last-mile secrets and protect your backend databases. Long living credentials need to be managed and regularly rotated to keep access into critical systems secure, so it’s a security best practice to periodically reset your passwords. Manually changing the passwords would be cumbersome, but AWS Secrets Manager helps by managing and rotating the RDS database passwords.
This is sample code: you’ll use an AWS CloudFormation template to deploy the following components to test the API endpoint from your browser:
- An RDS MySQL database instance on a db.t3.micro instance
- Two Lambda functions with necessary IAM roles and IAM policies, including access to AWS Secrets Manager:
- LambdaRDSCFNInit: This Lambda function will execute immediately after the CloudFormation stack creation. It will create an “Employees” table in the database, where it will insert three sample records.
- LambdaRDSTest: This function will query the Employees table and return the record count in an HTML string format
- RESTful API with “GET” method on AWS API Gateway
Here’s the high level setup of the AWS services that will be created from the CloudFormation stack deployment:
- Clients call the RESTful API hosted on AWS API Gateway
- The API Gateway executes the Lambda function
- The Lambda function retrieves the database secrets using the Secrets Manager API
- The Lambda function connects to the RDS database using database secrets from Secrets Manager and returns the query results
You can access the source code for the sample used in this post here: https://github.com/aws-samples/aws-secrets-manager-secure-database-credentials
Deploying the sample solution
Login into your AWS account, follow the prompts to log in.
By default, the stack will be deployed in the us-east-1 region. If you want to deploy this stack in any other region, download the code from the above source code link, build a zip file using the readme file, place the Lambda code zip file in a region-specific S3 bucket and make the necessary changes in the CloudFormation template to point to the right S3 bucket. (Please refer to the AWS CloudFormation User Guide for additional details on how to create stacks using the AWS CloudFormation console.)
Next, follow these steps to execute the stack:
- After login go to CloudFormation service in the AWS Console. Select on the Stacks option and then select Create stack option.
- Select Template is ready and then select Upload a template file. Next click on the Choose file button to choose the template file (secretsmanager_IaC.yml) from your desktop. Click Next.
- On the Specify stack details page, you’ll see the parameters pre-populated. These parameters include the name of the database and the database user name. Enter a name in the Stack name field. Select Next on this screen.
- On the Configure stack options page, select the Next button.
- On the Review screen, select all three check boxes, then select the Create change set button:
- After the change set creation is completed, choose the Execute button to launch the stack.
- Stack creation will take between 10 – 15 minutes. After the stack is created successfully, select the Outputs tab of the stack, then select the link.
This action will trigger the code in the Lambda function, which will query the “Employee” table in the MySQL database and will return the results count back to the API. You’ll see the following screen as output from the RESTful API endpoint:
At this point, you’ve successfully deployed and tested the API endpoint with a backend Lambda function and RDS resources. The Lambda function is able to successfully query the MySQL RDS database and is able to return the results through the API endpoint.
What’s happening in the background?
The CloudFormation stack deployed a MySQL RDS database with a randomly generated password using a secret resource. Now that the secret resource with randomly generated password has been created, the CloudFormation stack will use dynamic reference to resolve the value of the password from Secrets Manager in order to create the RDS instance resource. Dynamic references provide a compact, powerful way for you to specify external values that are stored and managed in other AWS services, such as Secrets Manager. The dynamic reference guarantees that CloudFormation will not log or persist the resolved value, keeping the database password safe. The CloudFormation template also creates a Lambda function to do automatic rotation of the password for the MySQL RDS database every 30 days. Native credential rotation can improve security posture, as it eliminates the need to manually handle database passwords through the lifecycle process.
Below is a reference CloudFormation code that covers these details (please make sure to use your version of the CloudFormation template as described in the sample code readme file):
Verifying the solution
To be certain that everything is set up properly, you can look at the Lambda code that’s querying the database table by following the below steps:
- Go to the AWS Lambda service page
- From the list of Lambda functions, click on the function with the name scm2-LambdaRDSTest-…
- You can see the environment variables at the bottom of the Lambda Configuration details screen. Notice that there should be no database password supplied as part of these environment variables:
In the AWS Secrets Manager console, you can also look at the new secret that was created from CloudFormation execution by following the below steps:
- Go to theAWS Secret Manager service page with appropriate IAM permissions
- From the list of secrets, click on the latest secret with the name MyRDSInstanceRotationSecret-…
- You will see the secret details and rotation information on the screen, as shown in the following screenshot:
In this post, we showed you how to manage database secrets using AWS Secrets Manager and how to leverage Secrets Manager’s API to retrieve the secrets into a Lambda execution environment to improve database security and protect sensitive data. Secrets Manager helps you protect access to your applications, services, and IT resources without the upfront investment and ongoing maintenance costs of operating your own secrets management infrastructure. To get started, visit the Secrets Manager console. To learn more, visit Secrets Manager documentation.
If you have feedback about this post, add it to the Comments section below. If you have questions about implementing the example used in this post, open a thread on the Secrets Manager Forum.
Want more AWS Security how-to content, news, and feature announcements? Follow us on Twitter.