Serving SSE-KMS encrypted content from S3 using CloudFront

Update:

1. We’ve updated this blog and the AWS Lambda function code to work with both “custom” and “s3” style origins in Amazon CloudFront. Previously, only “custom” types were covered.
2. In August 2022, CloudFront launched OAC (Origin Access Control), providing native support for customers to use CloudFront to access S3 bucket encrypted with SSE-KMS. Depending on your application use cases, you have the option to utilize either CloudFront OAC or Lambda@Edge to access S3 bucket encrypted with SSE-KMS. To learn more about using CloudFront OAC with SSE-KMS, please visit this blog.

Introduction

A best practice for your web applications is to use Amazon S3 to store content and Amazon CloudFront to deliver it to users. When building this way, AWS Well-Architected Framework recommends protecting your data at rest and in transit. Encryption is one of protection controls AWS provides you to reduce the risks of unauthorized access, loss, or exposure. In this blog post, you will quickly go through the available encryption options in S3 and CloudFront. Then, learn how to implement one of these options (SSE-KMS) in S3 when using CloudFront for content delivery.

Encryption options in S3 and CloudFront

With S3, you can either encrypt data at the client side and then upload the encrypted data to your S3 bucket, or to let S3 encrypt your data before storing it. The second method is called server-side encryption (SSE), and it comes in multiple flavors:

• Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3), where each object is encrypted with a unique key managed by S3
• Server-Side Encryption with Customer Master Keys (CMKs) stored in AWS Key Management Service (SSE-KMS). This gives you more control and visibility into how your encryption keys are being used
• Server-Side Encryption with customer-provided keys (SSE-C), where you manage the encryption keys and S3 only manages the encryption of objects

With CloudFront, you can encrypt data in transit using HTTPS, and enforce encryption policy by:

• Redirecting HTTP to HTTPS
• Choosing minimal TLS version and ciphers
• Selecting a domain name and its associated TLS certificate

Instead of exposing your S3 bucket publicly to allow CloudFront to download objects, it is best to keep your bucket private using CloudFront Origin Access Identity (OAI). OAI is a special CloudFront user that is associated with an S3 origin and given the necessary permissions to access to objects within the bucket. Currently, OAI only supports SSE-S3, which means customers cannot use SSE-KMS with OAI.

Enable SSE-KMS on S3 and serve content using CloudFront

Some organizations require you use SSE-KMS encryption on your S3 buckets and use CloudFront to deliver objects. In this section, you will learn how to serve content encrypted with SSE-KMS from S3 using CloudFront. Then, learn to use Lambda@Edge, a feature of CloudFront, to code custom logic on your CloudFront distribution using Javascript. Your Lambda@Edge functions are given IAM permissions to read from S3 and indirectly operate encryption/decryption using a CMK managed by KMS. These functions are triggered every time CloudFront makes a request to S3, and sign the request with AWS Signature Version 4 by adding the necessary headers. This signed request allows CloudFront to retrieve your object encrypted with SSE-KMS.

1 – Deploy a CloudFront distribution pointing to an S3 bucket with SSE-KMS enabled

First, deploy the CloudFormation template below in the Region of your choice. Make sure that the name of the stack is lowercase, otherwise the stack creation fails.

The corresponding CloudFormation stack includes:

• A CMK in KMS that can be used to encrypt and decrypt data by all users with S3 permissions
• An IAM role with permissions to manage the CMK
• An S3 bucket called [your-stack-name]-s3bucket with default bucket encryption set to SSE-KMS using the created CMK
• A CloudFront distribution using the bucket as the origin and OAI explicitly disabled.

AWSTemplateFormatVersion: '2010-09-09'
Resources:
  KMSAdmin:
    Type: AWS::IAM::Role
    Properties: 
      RoleName: !Join ['-', [!Ref 'AWS::StackName', 'KMSKey-admin']]
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              AWS: !Ref 'AWS::AccountId'
            Action:
              - 'sts:AssumeRole'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/AWSKeyManagementServicePowerUser'

  KMSKey:
    Type: 'AWS::KMS::Key'
    Properties:
      EnableKeyRotation: true
      KeyPolicy:
        Version: '2012-10-17'
        Statement:
          - Sid: Enable IAM User Permissions
            Effect: Allow
            Principal:
              AWS: !GetAtt KMSAdmin.Arn
              AWS: !Join ['', ['arn:aws:iam::', !Ref 'AWS::AccountId', ':root']]
            Action: 'kms:*'
            Resource: '*'
          - Sid: Allow access through S3 for all principals in the account that are authorized to use S3
            Effect: Allow
            Principal: 
              AWS: "*"
            Action:
            - kms:Encrypt
            - kms:Decrypt
            - kms:ReEncrypt*
            - kms:GenerateDataKey*
            - kms:DescribeKey
            Resource: '*'
            Condition:
              StringEquals:
                kms:CallerAccount: !Ref 'AWS::AccountId'
                kms:ViaService: !Join ['.', ['s3', !Ref 'AWS::Region', 'amazonaws.com']]
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Join ['-', [!Ref 'AWS::StackName', 's3bucket']]
      BucketEncryption:
        ServerSideEncryptionConfiguration: 
          - ServerSideEncryptionByDefault:
              KMSMasterKeyID: !Ref KMSKey
              SSEAlgorithm: aws:kms
  
              
  CloudFront:
    Type: 'AWS::CloudFront::Distribution'
    Properties:
      DistributionConfig:
        Comment: 'How to serve content encrypted with SSE-KMS from S3 using CloudFront'
        Origins:
        - DomainName: !Join ['.', [!Ref S3Bucket, 's3', !Ref 'AWS::Region', 'amazonaws.com']]
          Id: S3-regional-endpoint
          S3OriginConfig:
            OriginAccessIdentity: !Ref 'AWS::NoValue'
        DefaultCacheBehavior: 
          TargetOriginId: S3-regional-endpoint
          ForwardedValues:
            QueryString: 'false'
          ViewerProtocolPolicy: redirect-to-https
        Enabled: 'true'

2 – Add an object to the S3 bucket

Add a file to the S3 bucket for testing. Create the following HTML file, name it index.html, and upload it to S3.

<!DOCTYPE html>
<html>
<body>

<h1>How to serve content encrypted with SSE-KMS from S3 using CloudFront</h1>

<p>This HTML was served from an S3 bucket with SSE-KMS encryption enabled.</p>

</body>
</html>

Check the Server-side encryption attribute of this object in the Overview tab, and verify that it was encrypted by default by S3 with the KMS CMK.

For testing, force a cache miss and a fetch from S3 on every request CloudFront receives. This is done by setting the no-store directive in the Cache-Control header of the object.

This can be done in the object’s metadata in S3:

If you test the object URL using CloudFront, access is denied. We have not yet created the Lambda@Edge function that signs requests to S3, and allows CloudFront to retrieve the object.

3 – Create the Lambda@Edge function

Although Lambda@Edge runs on CloudFront’s global network, you must create the function in the N. Virginia Region (us-east-1). Go to the AWS Lambda console in us-east-1 and create a new function with the Node.js 12 runtime.

Choose create a new execution role, and select the Basic Lambda@Edge permissions policy template, then create the function.

Navigate to the created IAM role and attach the AWS managed policy named AmazonS3ReadOnlyAccess to the role. This allows the role (and the function) to sign requests to S3. You can enforce more restrictive permissions by allowing read access only to the specific bucket created in the stack. For more, read the documentation on security best practices with S3.

Copy the following code and paste it in the function using the embedded IDE in the Lambda console UI.

// Declare constants reqiured for the signature process
const crypto = require('crypto');
const emptyHash = 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855';
const signedHeadersGeneric = 'host;x-amz-content-sha256;x-amz-date;x-amz-security-token';
// CloudFront includes the x-amz-cf-id header in the signature for custom origins
const signedHeadersCustomOrigin = 'host;x-amz-cf-id;x-amz-content-sha256;x-amz-date;x-amz-security-token';
// Retrieve the temporary IAM credentials of the function that were granted by
// the Lambda@Edge service based on the function permissions. In this solution, the function
// is given permissions to read from S3 and decrypt using the KMS key.
const { AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN } = process.env;

// Since the function is configured to be executed on origin request events, the handler
// is executed every time CloudFront needs to go back to the origin, which is S3 here.
exports.handler = async event => {

    // Retrieve the original request that CloudFront was going to send to S3
    const request = event.Records[0].cf.request;

    // The request object has different properties depending on the type of
    // origin that is being used. Account for that here.
    let originType = '';
    if (request.origin.hasOwnProperty('s3'))
        originType = 's3';
    else if (request.origin.hasOwnProperty('custom'))
        originType = 'custom';
    else
        throw("Unexpected origin type. Expected 's3' or 'custom'. Got: " + JSON.stringify(request.origin));

    // Create a JSON object with the fields that should be included in the Sigv4 request,
    // including the X-Amz-Cf-Id header that CloudFront adds to every request forwarded
    // upstream. This header is exposed to Lambda@Edge in the event object
    const sigv4Options = {
        method: request.method,
        path: request.origin[originType].path + request.uri,
        credentials: {
            accessKeyId: AWS_ACCESS_KEY_ID,
            secretAccessKey: AWS_SECRET_ACCESS_KEY,
            sessionToken: AWS_SESSION_TOKEN
        },
        host: request.headers['host'][0].value,
        xAmzCfId: event.Records[0].cf.config.requestId,
        originType: originType
    };

    // Compute the signature object that includes the following headers: X-Amz-Security-Token, Authorization,
    // X-Amz-Date, X-Amz-Content-Sha256, and X-Amz-Security-Token
    const signature = signV4(sigv4Options);

    // Finally, add the signature headers to the request before it is sent to S3
    for(var header in signature){
        request.headers[header.toLowerCase()] = [{
            key: header,
            value: signature[header].toString()
        }];
    }

    return request;
};


// Helper functions to sign the request using AWS Signature Version 4
// This helper only works for S3, using GET/HEAD requests, without query strings
// https://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
function signV4(options) {
    // Infer the region from the host header
    const region = options.host.split('.')[2];
    // Create the canonical request
    const date = (new Date()).toISOString().replace(/[:-]|\.\d{3}/g, '');
    let canonicalHeaders = '';
    let signedHeaders = '';
    if (options.originType == 's3') {
        canonicalHeaders = ['host:'+options.host, 'x-amz-content-sha256:'+emptyHash, 'x-amz-date:'+date, 'x-amz-security-token:'+options.credentials.sessionToken].join('\n');
        signedHeaders = signedHeadersGeneric;
    } else {
        canonicalHeaders = ['host:'+options.host, 'x-amz-cf-id:'+options.xAmzCfId, 'x-amz-content-sha256:'+emptyHash, 'x-amz-date:'+date, 'x-amz-security-token:'+options.credentials.sessionToken].join('\n');
        signedHeaders = signedHeadersCustomOrigin;
    }
    const canonicalURI = encodeRfc3986(encodeURIComponent(decodeURIComponent(options.path).replace(/\+/g, ' ')).replace(/%2F/g, '/'));
    const canonicalRequest = [options.method, canonicalURI, '', canonicalHeaders + '\n', signedHeaders,emptyHash].join('\n');
    // Create string to sign
    const credentialScope = [date.slice(0, 8), region, 's3/aws4_request'].join('/');
    const stringToSign = ['AWS4-HMAC-SHA256', date, credentialScope, hash(canonicalRequest, 'hex')].join('\n');
    // Calculate the signature
    const signature = hmac(hmac(hmac(hmac(hmac('AWS4' + options.credentials.secretAccessKey, date.slice(0, 8)), region), "s3"), 'aws4_request'), stringToSign, 'hex');
    // Form the authorization header
    const authorizationHeader = ['AWS4-HMAC-SHA256 Credential=' + options.credentials.accessKeyId + '/' + credentialScope,'SignedHeaders=' + signedHeaders,'Signature=' + signature].join(', ');

    // return required headers for Sigv4 to be added to the request to S3
    return {
        'Authorization': authorizationHeader,
        'X-Amz-Content-Sha256' : emptyHash,
        'X-Amz-Date': date,
        'X-Amz-Security-Token': options.credentials.sessionToken
    };
}

function encodeRfc3986(urlEncodedStr) {
  return urlEncodedStr.replace(/[!'()*]/g, c => '%' + c.charCodeAt(0).toString(16).toUpperCase())
}

function hash(string, encoding) {
  return crypto.createHash('sha256').update(string, 'utf8').digest(encoding)
}

function hmac(key, string, encoding) {
  return crypto.createHmac('sha256', key).update(string, 'utf8').digest(encoding)
}

Finally, deploy this function to the CloudFront distribution for origin request events by clicking on the Actions menu, then the Deploy to Lambda@Edge option. Make sure that you use the distribution created by the CloudFormation stack. The origin request event is triggered every time CloudFront makes a request upstream to the origin, in this case S3.

4 – Test the solution

Wait for a couple of minutes until the distribution is deployed, and then test the object URL again. It should work now! Congrats!

It’s important to know the median execution time of your function. This has consequences for the cost of the solution and its latency overhead. In our tests, we sent requests in a loop for an hour to trigger the function. The median execution duration recorded in CloudWatch was around 1 ms.

Further thoughts

In the previous section, you served content encrypted with SSE-KMS from S3 using CloudFront. You may ask, how about the other direction, uploading content to S3 using CloudFront and encrypting it with SSE-KMS? Here are some hints for how to implement this option:

• When you upload an object to S3, the request body is included in the signature, which means that Lambda@Edge has to access the request body. To do that, make sure to enable the Include Body option when you deploy the Lambda@Edge function. Note, the current maximum body size that can be passed to Lambda@Edge is 1 MB for origin request events.
• In the signature, make sure you add the Content-Type header as well as the body when the request is PUT or POST.
• Don’t forget to allow PUT/POST requests in CloudFront, and to update the Lambda@Edge function execution role permissions to allow it to write to S3.

Finally, the helper function (signV4) was provided in the code to sign requests in a simple but basic way. For your production environment, you might want to consider a more robust and flexible library such as aws4.

Cleanup

When you validate this proof of concept, clean up the created resources to avoid incurring costs. Delete the index.html file from the S3 bucket, then delete the deployed CloudFormation stack. For more information, see Deleting a Stack in the AWS documentation. Finally, delete the Lambda@Edge function and the associated IAM role.

Conclusion

In this solution, each time CloudFront fetches your object from S3 a Lambda@Edge function is executed and signs the request correctly using AWS Signature Version 4. As a next step, please consider the costs and quotas of Lambda@Edge with regard to this solution, as explained in previous blogs.

Finally, you can go beyond S3 as an origin, and use this signing method to securely access other AWS services directly from CloudFront.