AWS Database Blog

IAM role-based authentication to Amazon Aurora from serverless applications

January 2024: This post was reviewed and updated for accuracy.

Storing user names and passwords directly in applications is not a best practice. Saving credentials as plaintext should never occur in a secure application. As a solution, AWS Identity and Access Management (IAM) policies can assign permissions that determine who is allowed to manage Amazon Aurora resources. For example, you can use IAM to determine who is allowed to create, describe, modify, and delete DB clusters, tag resources, or security groups. In Amazon Aurora, you can associate the database users with the IAM user and roles.

In this post, I explain how to use IAM for accessing your Amazon Aurora databases using an AWS Lambda function.

Overview

You can use IAM database authentication to connect to the DB cluster. Using this method, you can access the database with an authentication token generated instead of storing the password in a configuration file. Amazon Aurora generates an AWS Signature Version 4 authentication token that is valid for 15 minutes to create a connection from your application. As authentication is fully managed externally by IAM, you do not need to create credentials in the database.

Walkthrough

The following diagram shows the workflow.

Lambda is a compute service that runs your code without provisioning or managing servers. Lambda executes your code only when needed and scales automatically, from a few requests per day to thousands per second. When using Lambda, you are responsible only for your code. Lambda manages the compute fleet that offers a balance of memory, CPU, network, and other resources. Lambda executes the Lambda function and returns results.

Here’s how you can create a Lambda function with the console.

  • Open the AWS Lambda console.
  • Choose Create function.
  • For Function name, enter demo-function.
  • Select your preferred language runtime. For this post, choose Python 3.11.
  • For permissions, select Create a new role with basic Lambda permission. This creates an execution role (for example, demo-function-role-ipqlab4h), with permissions to upload logs to Amazon CloudWatch
  • When you choose Create function, you see the options shown in the following screenshot.

Designer lets you configure triggers and view permissions.

You can create a function using the AWS Lambda console editor, as shown in the following screenshot.

Lambda includes sample code that returns a success response. The code editor in the Lambda console enables you to write, test, and view the execution results of your Lambda function code.

You can edit your function code with the embedded AWS Cloud9 editor as long as your source code doesn’t exceed the 3 MB limit. If you must include any libraries and dependencies other than the AWS SDK, you must create a deployment package. You can upload the package directly if it does not exceed 50 MB. Otherwise, you can upload to Amazon S3 and then upload it to Lambda.

The following screenshot shows a dialog box allowing you to set Lambda environment variables.

Environment variables for Lambda functions enable you to dynamically pass settings to your function code and libraries, without making changes to your code.

The following screenshot shows a dialog allowing you to set a Lambda execution role:

A Lambda function’s execution role grants it permission to access AWS services and resources.

The following screenshot shows a dialog box allowing you to choose a network:

By default, a Lambda function has access to your public internet address and public AWS APIs. To access the private resources like Amazon RDS located in private subnets, you must enable your Lambda function for VPC. It’s a best practice to configure the Lambda function in the same VPC where the database instance resides. Additionally, ensure that the private subnets have a NAT gateway added in their route tables to grant the Lambda function internet access.

Here’s how you can access Amazon Aurora from a Lambda function using IAM authentication:

  • You need the AWS Command Line Interface (AWS CLI) installed and configured on your machine before moving to the next steps.
  • Enable IAM database authentication in the DB cluster using the AWS CLI as follows. You could also use the AWS Management Console or Amazon RDS API. 
    aws rds modify-db-cluster --db-cluster-identifier <cluster-identifier> --enable-iam-database-authentication --apply-immediately
    • cluster-identifier: name of your DB cluster.
      Example:

      aws rds modify-db-cluster --db-cluster-identifier mysql-demo  --enable-iam-database-authentication --apply-immediately
  • Connect to the DB cluster, and create a user with login privileges and grant IAM role access to the user:
    • PostgreSQL: Grant rds_iam privilege to the user. 
      CREATE USER <db_user_name> WITH LOGIN; 
GRANT rds_iam TO <db_user_name>;

      Example:

      CREATE USER  demouser WITH LOGIN; 
GRANT rds_iam TO demouser;
    • MySQL: Grant privileges as follows. 
      CREATE USER < db_user_name> IDENTIFIED WITH AWSAuthenticationPlugin as 'RDS';
GRANT ALL PRIVILEGES ON <dbname>.* TO '<db_user_name>’ @'%';

      Example:

      CREATE USER demouser IDENTIFIED WITH AWSAuthenticationPlugin as 'RDS';
GRANT ALL PRIVILEGES ON demodb.* TO 'demouser’ @'%';
FLUSH PRIVILEGES;
  • Now, create the policy which allows the user created in the previous step to access the database using IAM database authentication. For more information, see IAM Policy and IAM Database Access.
    Policy Document:

    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "rds-db:connect"
            ],
            "Resource": [
                "arn:aws:rds-db:<region>:<account-id>:dbuser:<DbiResourceId>/<db_user_name>"
            ]
        }
    ]
}

    Use the following settings.

    • region: The region of the database instance.
    • account-id: The AWS account number of the database instance.
    • DbiResourceId: The DB instance identifier, or * to allow all the database instances in the region of the account.
    • db_user_name: The user that has been created in the database. Mention a specific user, or * to allow IAM authentication for all users in the database.

    Follow this example to create the policy using AWS CLI.

    aws iam create-policy --policy-name my-policy --policy-document '{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "rds-db:connect"
            ],
            "Resource": [
                "arn:aws:rds-db:us-east-1:123456789012:dbuser:*/demouser"
            ]
        }
    ]
}'
  • Now attach this policy to the IAM role associated with the Lambda function: 
    aws iam attach-role-policy --role-name <role_name> --policy-arn <policy_arn>
    • role_name: IAM role associated with the Lambda function (for example, demo-function-role-ipqlab4h).
    • policy_arn: ARN of IAM policy created for IAM database authentication (for example  “arn:aws:iam::123456789012:policy/my-policy”).
  • Set the following environment variables in Lambda.
    • DBEndPoint: The DB instance endpoint (for example, ab9c3efgh31k.us-east-1.rds.amazonaws.com).
    • DatabaseName: The name of the database.
    • DBUserName: The user name created in the previous steps (for example, demouser).

To encrypt the network traffic, use the RDS-provided root SSL certificate, such as this example. You must include this SSL certificate in your deployment package.

To access the Amazon Aurora PostgreSQL database from your Lambda function, you must include the pg8000 (PostgreSQL Driver) library as part of your deployment package.

Here is the Lambda function for Amazon Aurora PostgreSQL.

import boto3
import pg8000.dbapi
import os
import ssl
import logging

logger = logging.getLogger()

# declare the global connection object to use during warm starting
# to reuse connections that were established during a previous invocation.
connection = None

def get_connection():
    """
        Method to establish the connection.
    """
    try:
        logger.info ("Connecting to database")
        # Create a low-level client with the service name for rds
        client = boto3.client("rds")
        # Read the environment variables to get DB EndPoint
        DBEndPoint = os.environ.get("DBEndPoint")
        # Read the environment variables to get the Database name
        DatabaseName = os.environ.get("DatabaseName")
        # Read the environment variables to get the Database username which has access to database.
        DBUserName = os.environ.get("DBUserName")
        # ssl context information
        ssl_context = ssl.create_default_context()
        ssl_context.verify_mode = ssl.CERT_REQUIRED
        # update the certificate filename based on the one which is packaged with lambda function.
        ssl_context.load_verify_locations('us-east-1-bundle.pem')
        # Generates an auth token used to connect to a db with IAM credentials.
        password = client.generate_db_auth_token(
            DBHostname=DBEndPoint, Port=5432, DBUsername=DBUserName
        )
        # Establishes the connection with the server using the token generated as password
        conn = pg8000.dbapi.connect(
            host=DBEndPoint,
            user=DBUserName,
            database=DatabaseName,
            password=password,
            ssl_context=ssl_context
        )
        return conn
    except Exception as e:
        logger.error ("While connecting failed due to :{0}".format(str(e)))
        return None

def lambda_handler(event, context):
    global connection
    try:
        if connection is None:
            connection = get_connection()
        if connection is None:
            return {"status": "Error", "message": "Failed"}
        logger.info ("instantiating the cursor from connection")
        # Instantiate the cursor object
        cursor = connection.cursor()
        # Query to be executed
        query = "SELECT CURRENT_DATABASE()"
        # execute the query / command in the connected database
        cursor.execute(query)
        # Get the column names
        columns = [str(desc[0]) for desc in cursor.description]
        results = []
        # create the rows of dict from the result set.
        for res in cursor:
            results.append(dict(zip(columns, res)))
        cursor.close()
        # result to return
        response = {"status": "Success", "results": results}
        return response
    except Exception as e:
        try:
            connection.close()
        except Exception as e:
            connection = None
            logger.error ("Failed due to :{0}".format(str(e)))
        return {"status": "Error", "message": "Something went wrong. Try again"}

To access Amazon Aurora MySQL database from your Lambda function, you must include mysql-connector-python (MySQL Driver) library as part of your deployment package.

Here is the Lambda function for Amazon Aurora MySQL.

import os
import boto3
from mysql import connector
from mysql.connector import Error
import logging

logger = logging.getLogger()

# declare the global connection object to use during warm starting
# to reuse connections that were established during a previous invocation.
connection = None

def get_connection():
    """
        Method to establish the connection.
    """
    try:
        logger.info ("Connecting to database")
        # Create a low-level client with the service name for rds
        client = boto3.client("rds")
        # Read the environment variables to get DB EndPoint
        DBEndPoint = os.environ.get("DBEndPoint")
        # Read the environment variables to get the Database name
        DatabaseName = os.environ.get("DatabaseName")
        # Read the environment variables to get the Database username which has access to database.
        DBUserName = os.environ.get("DBUserName")
        # Generates an auth token used to connect to a db with IAM credentials.
        password = client.generate_db_auth_token(
            DBHostname=DBEndPoint, Port=3306, DBUsername=DBUserName
        )
        # Establishes the connection with the server using the token generated as password
        conn = connector.connect(
            host=DBEndPoint,
            database=DatabaseName,
            user=DBUserName,
            password=password,
            auth_plugin="mysql_clear_password",
            ssl_ca="us-east-1-bundle.pem",
        )
        return conn
    except Exception as e:
        logger.error ("While connecting failed due to :{0}".format(str(e)))
        return None

def lambda_handler(event, context):
    global connection
    try:
        if connection is None:
            connection = get_connection()
        if connection is None:
            return {"status": "Error", "message": "Failed"}
        logger.info ("instantiating the cursor from connection")
        # Instantiate the cursor object
        # Instantiate the cursor object with dictionary as true to return the rows as dictionaries
        cursor = connection.cursor(dictionary=True)
        # execute the query / command in the connected database
        cursor.execute("select database() as database_name")
        # Retreive the next row of query result set.
        record = cursor.fetchone()
        # Return the results
        response = {"status": "Success", "results": record}
        return response
    except Exception as e:
        try:
            connection.close()
        except Exception as e:
            connection = None
        logger.error ("Failed due to :{0}".format(str(e)))
        return {"status": "Error", "message": "Something went wrong. Try again"}

Conclusion

Many applications store the database credentials in the plaintext files, which are not encrypted and expose credentials to the users, which is a potential risk. Now you can use IAM to centrally manage access to your database resources, instead of managing access individually on each database instance. As long as your IAM user or role has the permission to access the database, you can generate the authentication token to access the database.

About the Author

Mahesh Balumuri is a Consultant with AWS Professional Services. He works with AWS customers to provide guidance and technical assistance on database projects, helping them to build their infrastructure and applications on the cloud. He also has a good knowledge of automation, orchestration, and DevOps in cloud environments.