How can I use an AWS CloudFormation resource import to create an Amazon S3 notification configuration for AWS Lambda on an existing S3 bucket?

Last updated: 2020-09-03

I want to create an Amazon Simple Storage Service (Amazon S3) notification configuration for AWS Lambda on an existing S3 bucket by importing a resource using AWS CloudFormation.

Short description

To configure an Amazon S3 notification without using a custom resource, you can do the following:

  1. Create a template with the Lambda function S3NotificationLambdaFunction, which adds the existing bucket NotificationBucket notification configuration.
  2. Use a resource import to bring the existing S3 bucket NotificationBucket (specified in the template that you created) into AWS CloudFormation management.
  3. Update the AWS CloudFormation stack to include the properties that you want to enable in your S3 bucket.

To use a custom resource, see How can I create an Amazon S3 notification configuration for Lambda on an existing S3 bucket using AWS CloudFormation?

Resolution

Important: The following steps override any existing or manually created notification configurations in an S3 bucket. You can follow these steps to add a new notification configuration to an imported S3 bucket.

Create an AWS CloudFormation template with a Lambda function

The following example template creates a Lambda function, with an executing role and permissions to invoke the function. To use an existing Lambda function, use the Lambda function Amazon Resource Name (ARN) in your AWS CloudFormation template for the LambdaConfiguration property in the S3 bucket.

AWSTemplateFormatVersion: 2010-09-09
Description: >-
  Sample template to illustrate use of importing existing S3 bucket as event source for Lambda function
Parameters:
  NotificationBucket:
    Type: String
    Description: S3 bucket which is used for Lambda event notification
Resources:
  S3NotificationLambdaFunction:
    Type: 'AWS::Lambda::Function'
    Properties:
      Code:
        ZipFile: !Join
          - |+

          - - import json
            - 'def lambda_handler(event,context):'
            - '    return ''Welcome... This is a test Lambda Function'''
      Handler: index.lambda_handler
      Role: !GetAtt LambdaIAMRole.Arn
      Runtime: python3.6
      Timeout: 5
  LambdaInvokePermission:
    Type: 'AWS::Lambda::Permission'
    Properties:
      FunctionName: !GetAtt S3NotificationLambdaFunction.Arn
      Action: 'lambda:InvokeFunction'
      Principal: s3.amazonaws.com
      SourceAccount: !Ref 'AWS::AccountId'
      SourceArn: !Sub 'arn:aws:s3:::${NotificationBucket}'
  LambdaIAMRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      Policies:
        - PolicyName: root
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 's3:GetBucketNotification'
                  - 's3:PutBucketNotification'
                Resource: !Sub 'arn:aws:s3:::${NotificationBucket}'
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Resource: 'arn:aws:logs:*:*:*'

Import an existing S3 bucket to your AWS CloudFormation stack

1.    Open the AWS CloudFormation console.

2.    From the navigation pane, choose Stack, and then select the stack that you created earlier.

3.    Choose Stack actions, and then choose Import resources into stack.

4.    Review the Import overview page, and then choose Next.

Important: In your AWS CloudFormation template, each resource that you import must have a DeletionPolicy attribute and all other resources must remain the same. See the following example:

AWSTemplateFormatVersion: 2010-09-09
Description: >-
  Sample template to illustrate the use of importing an existing S3 bucket as an event source for a Lambda function
Parameters:
  NotificationBucket:
    Type: String
    Description: S3 bucket which is used for Lambda event notification
Resources:
<Excluded the resources as the below resource is addition to above resources provided in step(a)>
   NotificationBucket:
      Type: 'AWS::S3::Bucket'
      DeletionPolicy: Retain            
      Properties:
        BucketName: myenv-bucket          #Bucket name to import

5.    In the Specify template section, choose Amazon S3 URL or Upload a template file based on your requirements, and then choose Next.

6.    Complete the rest of the steps in the wizard to import your existing resources. For more information, see Import an existing resource into a stack using the AWS CloudFormation console.

7.    Wait for the stack to reach the IMPORT_COMPLETE state.

Update your AWS CloudFormation template

1.    Update the stack with the AWS CloudFormation template that you modified earlier.

2.    Wait for the stack to reach the UPDATE_COMPLETE state, and then verify the NotificationConfiguration on the S3 bucket.

3.    Enable Amazon S3 event notifications on your S3 bucket (myenv-bucket in the following example). For example:

AWSTemplateFormatVersion: 2010-09-09
Description: >-
  Sample template to illustrate use of importing existing S3 bucket as event source for
  Lambda function
Parameters:
  NotificationBucket:
    Type: String
    Description: S3 bucket which is used for Lambda event notification
Resources:
<Excluded the resources as the below resource is addition to above resources provided in step(a)>
    NotificationBucket:
      Type: 'AWS::S3::Bucket'
      DeletionPolicy: Retain            
      Properties:
        BucketName: myenv-bucket
        NotificationConfiguration:
          LambdaConfigurations:
              - Event: 's3:ObjectCreated:Put'
                Function: !GetAtt S3NotificationLambdaFunction.Arn
              - Event: 's3:ObjectRemoved:*'
                Function: !GetAtt S3NotificationLambdaFunction.Arn

Did this article help?


Do you need billing or technical support?