Securing and Accessing Secrets from Lambda@Edge using AWS Secrets Manager

Lambda@Edge is a feature of Amazon CloudFront that lets you run code closer to users of your application, across on the globe, improving performance and reducing latency. This feature is useful for enriching HTTP requests with filters, security headers, and dynamically routing a request to a specific origin. When working with Lambda@Edge, there are situations where you must access sensitive or secure values. For example, when securing your CloudFront distributions using Lambda@edge and want to store the secure values for OpenID Connect. To learn more about edge networking on AWS, click here. This blog covers a specific use case for Lambda@Edge, calling an external service API right before routing the end user to the request origin. As it is not a good practice to store the API keys in the code, one option is to use AWS Secrets Manager to securely store the API keys. Those keys are then retrieved by the Lambda@Edge function to complete the request.

What You Will Build

In this blog, you will create a Lambda@Edge function that stores and retrieves an API Key Secret from AWS Secrets Manager. You use this API key to invoke the HTTP POST call to an external web service. Then, you learn how to use AWS Lambda variables to store values for multiple invocations in different Regions, with reduced latency.

Pre-requisites

Before continuing on, it is assumed that you have already done the following:
1.     Create an AWS account. You can start that process here.
2.     Create an Amazon CloudFront distribution. Instructions can be found here.

Architecture Design

In the preceding architecture, the following is occurring:
1.     The end user hits the Amazon CloudFront distribution endpoint.
2.     An Amazon CloudFront viewer request was invoked at an edge location.
3.     The Lambda@Edge function retrieves the appropriate secret(s) from AWS Secrets Manager.

The following approach uses a single AWS Region for Secrets Manager. It uses a  Lambda global variable to cut down on the number of times Lambda@Edge must reach out to Secrets Manager. (A global variable is a value that is defined outside the scope of the Lambda handler function and is globally available within the scope of the execution.) This value is re-used and set for each invocation. If it is the first time the Lambda function is executed, it reaches out to gather the secrets – otherwise it uses the global variable. In this example, you use one AWS Region to store your secure values in Secrets Manager.

Lambda@Edge IAM Execution Role

First, create an IAM role that allows your Lambda@Edge function:
1.     Log events and messages to CloudWatch.
2.     Access your secret from AWS Secrets Manager.
3.     Decrypt the secret using the AWS Key Management Service (AWS KMS) key.

Grant your Lambda@Edge function access:
1.      Sign in to the AWS Management Console and navigate to IAM
2.     Create a new role by selecting Lambda from the list of services.
3.     Select the Next: Permissions button
4.     On the next page, attach the managed policy AWSLambdaBasicExecutionRole.
5.     Name the role “LambdaEdgeExampleRole”.

IAM Trust Policy

In order to allow Lambda@Edge to assume and use the role created above, modify the Trust Relationship of the role.

1.     Navigate back to the IAM role and click on Trust Relationships and then Edit trust relationship.
2.     Your trust relationship document should look like the following. (This allows the edgelambda and lambda service  roles access to use your Lambda@Edge execution role created from above.)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    },
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "edgelambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Pre-requisites for creating a secret in AWS Secrets Manager

To secure your API key at rest in AWS Secrets Manager, create an AWS KMS Key.
You can find instructions on how to create an AWS KMS key here

Once the key is created, provide it with a key policy:

Secrets Manager KMS Key Policy:

{
    "Id": "secrets-manager-kms-key-policy",
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Enable IAM User Permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::012345678910:root"
            },
            "Action": "kms:",
            "Resource": "*"
        },
        {
            "Sid": "Allow access for Key Administrators",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::012345678910:role/LambdaEdgeExampleRole"
            },
            "Action": [
                "kms:Create",
                "kms:Describe",
                "kms:Enable",
                "kms:List",
                "kms:Put",
                "kms:Update",
                "kms:Revoke",
                "kms:Disable",
                "kms:Get",
                "kms:Delete",
                "kms:TagResource",
                "kms:UntagResource",
                "kms:ScheduleKeyDeletion",
                "kms:CancelKeyDeletion"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow use of the key",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::012345678910:role/LambdaEdgeExampleRole"
            },
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt",
                "kms:GenerateDataKey",
                "kms:DescribeKey"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow attachment of persistent resources",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::012345678910:role/LambdaEdgeExampleRole"
            },
            "Action": [
                "kms:CreateGrant",
                "kms:ListGrants",
                "kms:RevokeGrant"
            ],
            "Resource": "*",
            "Condition": {
                "Bool": {
                    "kms:GrantIsForAWSResource": "true"
                }
            }
        }
    ]
}

Creating a secret in Secrets Manager

Navigate back to the AWS Management Console and search for Secrets Manager. After navigating to the service page, click on Store a new secret. Once on this page, you see the following screen:

Select Other type of secrets for storing your API key. This secret type is a simple key-value pair. In this example, the name of the secret key is apikey. The value of the key is the actual value of the API key (here it is set to API_KEY_VALUE). Next, select the AWS KMS key that was created in the preceding step. Finally, select Next.

In this step, you will provide a name for the secret. In this example, the name of the secret is ApiKey.

In the final step, select Store. This stores the Secret in Secrets Manager. Congratulations!

IAM Policy for AWS Secrets Manager Access

Now, create a new IAM Policy that allows this role access to read a secret out of AWS Secrets Manager.
Copy down the ARN of the secret you created above, you need to specify this in the Resource section of the policy.

Here is the JSON policy document that allows the Lambda@Edge role access to read the secret  from AWS Secrets Manager:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "secretsmanager:GetSecretValue",
            "Resource": "arn:aws:secretsmanager:us-east-1:012345678910:secret:app/api/key-123xyz"
        }
    ]
}

Name the policy LambdaEdgeSecretsManagerExamplePolicy and attach it to the role LambdaEdgeExampleRole that was created above.

Lambda@Edge Function

The Python code below is an example of a Lambda@Edge function that reaches out to AWS Secrets Manager to retrieve an API key. Use this API key to invoke the HTTP POST call to an external web service.

edge_api_call.py:

import sys
import json
import boto3
import base64
from botocore.vendored import requests
from botocore.exceptions import ClientError

# global variables for re-use of invocations
session = boto3.session.Session()
apiEndpoint = "https://myapiendpoint/api/v1"
secretName = "app/api/key"
secretRegion = "us-east-1"

# global variable to cut down the number of calls to AWS Secrets Manager
apiKey = ""

def lambda_handler(event, context):
    """
    lambda handler function for invocation.
    """
    
    # pull the API key from AWS Secrets Manager
    # only reach out to grab key if it is an empty string
    if apiKey == "":
        apiKey = get_secret(secretName, secretRegion)['apikey']
    else:
        print("apiKey is already set, moving on")
    
    # HTTP POST call to external service
    apiresponse = call_api_endpoint(
        endpoint=apiEndpoint,
        payload={"source": "lambda-edge"},
        headers={"Authorization": apiKey}
    )
    
    if apiresponse.return_code == '200':
        return {
         'status': '200',
         'statusDescription': 'Good'
     }
    else:
        return {
         'status': apiresponse.return_code,
         'statusDescription': 'Error'
     }
    
    
def call_api_endpoint(
    endpoint: str, 
    payload: dict,
    headers: dict
):
    """
    Makes HTTP POST call to service endpoint.

    Parameters:
        endpoint (str) = the HTTP endpoint to call.
        payload (dict) = the payload to pass to the endpoint.
        headers (dict) = all headers to pass to the HTTP request.
    
    Returns:
        response = the HTTP response.
    """

    try:
        response = requests.post(
            url=endpoint,
            data=payload,
            headers=headers
        )
        return response
    except requests.exceptions.HTTPError:
        print("An exception occurred while calling the API endpoint!")
        sys.exit(1)
        
def get_secret(secret_name: str, region_name: str):
    """
    Gets the secret from AWS Secrets Manager.
    
    Parameters:
        secret_name (str) = the name of the secret to get.
        region_name (str) = the name of the AWS region where the secret is.
        
    Returns:
        secret_val (dict) = the key-value pair of the secret.
    """
    
    client = session.client(
        service_name='secretsmanager',
        region_name=region_name
    )

    try:
        get_secret_value_response = client.get_secret_value(
            SecretId=secret_name
        )
    except ClientError as e:
            raise e
        else:
            # Decrypts secret using the associated KMS CMK.
            # Depending on whether the secret is a string or binary, one of these fields will be populated.
            if 'SecretString' in get_secret_value_response:
                secret = get_secret_value_response['SecretString']
                return json.loads(secret)
            else:
                decoded_binary_secret = base64.b64decode(get_secret_value_response['SecretBinary'])
                return json.loads(decoded_binary_secret)

Breakdown of function code

The code above is broken into a couple of functions:

1.     lambda_handler = this function is the entrypoint of the Lambda@Edge function and gets invoked on a viewer request. This function ties together all steps to retrieve an API key from AWS Secrets Manager and perform an HTTP POST call to an external API.
2.     call_api_endpoint = this function performs an HTTP POST call to an external API.
3.     get_secret = this function reaches out to AWS Secrets Manager and retrieves the secret API key.

The following global variables are defined:

1.     session = this is a reusable AWS boto3 session object that is used to create boto3 client objects to interact with AWS services.
2.     apiEndpoint = this is the external HTTP API endpoint that the Lambda@Edge function makes a POST call to.
3.     secretName = this is the name of the secret in AWS Secrets Manager to retrieve.
4.     secretRegion = this is the AWS Region where the secret lives.

NOTE: ApiEndpoint is not a real HTTP API endpoint and will not call any real service. This is meant for demonstrative purposes.

One important thing to note is the following lines of code from above:

# pull the API key from AWS Secrets Manager
    # only reach out to grab key if it is an empty string
    if apiKey == "":
        apiKey = get_secret(secretName, secretRegion)['apikey']
    else:
        print("apiKey is already set, moving on")

This block of code checks to see if the variable apiKey is an empty string. If the value of this variable is non-empty, you do not need to reach back out to AWS Secrets Manager to retrieve it.

Creating and deploying the Lambda@Edge function

Navigate to the AWS Management Console in us-east-1 Region and search for Lambda. Once on this page, create a new function and choose Python3.7 for the runtime. Choose the IAM role that was created in the previous step, that is, the IAM role with the name LambdaEdgeExampleRole. After creating the new Lambda Function, paste the code from above in the function contents and click save.

Finally, navigate to Actions and select Deploy to Lambda@Edge as shown below:

After you click on Deploy to Lambda@Edge, you will be asked to choose or create a new CloudFront trigger. Select Configure new CloudFront trigger and select the distribution you created before this blog with the correct cache behavior. Be sure to configure the CloudFront event as a Viewer request. Finally, acknowledge you are OK with this function being replicated to all AWS Regions. You are now done!

Conclusion

In this blog, you successfully set up a Lambda@Edge function that securely and efficiently reaches out to AWS Secrets Manager to retrieve API keys. You set up the required IAM roles, policies, and KMS key policies that allow your Lambda@Edge execution role the ability to access secrets in a secure manner. You have learned about Lambda global variables and how this helps cut down on the number of calls Lambda@Edge must make to AWS Secrets Manager.