How can I create an Amazon S3 notification configuration for Lambda on an existing S3 bucket using AWS CloudFormation?
Last updated: 2020-07-08
I want to use an existing Amazon Simple Storage Service (Amazon S3) bucket to create an Amazon S3 notification configuration for an AWS Lambda function using AWS CloudFormation.
Short description
To create an Amazon S3 notification configuration, you can use AWS CloudFormation to create a new S3 bucket. Then, add a notification configuration to that bucket using the NotificationConfiguration property. Or, manually add a notification configuration to an existing S3 bucket.
The following steps show you how to add a notification configuration to your existing S3 bucket with AWS CloudFormation by using a Lambda-backed custom resource created in Python 3.6. The custom resource triggers a Lambda function, which triggers the PutBucketNotification API to add a notification configuration to your S3 bucket.
Resolution
Important: The following steps apply only to S3 notification configurations for S3 buckets that don't have any existing notification configurations. If your S3 bucket already has an existing or manually-created notification configuration, the following steps override those configurations.
1. Create an AWS CloudFormation template called LambdaS3.template that includes the following code.
Important: In the following example, you add the S3 notification configuration to the S3NotificationLambdaFunction resource. You use the Lambda function CustomResourceLambdaFunction to add the S3 notification configuration for S3NotificationLambdaFunction. You can modify the code in the CustomResourceLambdaFunction resource to meet your requirements.
AWSTemplateFormatVersion: 2010-09-09
Description: >-
Sample template to illustrate use of existing S3 bucket as an event source for a Lambda function
Parameters:
NotificationBucket:
Type: String
Description: S3 bucket that's used for the Lambda event notification
Resources:
S3NotificationLambdaFunction:
Type: 'AWS::Lambda::Function'
Properties:
Code:
ZipFile: !Join
- |+
- - import json
- 'def lambda_handler(event,context):'
- ' return ''Welcome... This is a test Lambda Function'''
Handler: index.lambda_handler
Role: !GetAtt LambdaIAMRole.Arn
Runtime: python3.6
Timeout: 5
LambdaInvokePermission:
Type: 'AWS::Lambda::Permission'
Properties:
FunctionName: !GetAtt S3NotificationLambdaFunction.Arn
Action: 'lambda:InvokeFunction'
Principal: s3.amazonaws.com
SourceAccount: !Ref 'AWS::AccountId'
SourceArn: !Sub 'arn:aws:s3:::${NotificationBucket}'
LambdaIAMRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- 'sts:AssumeRole'
Path: /
Policies:
- PolicyName: root
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 's3:GetBucketNotification'
- 's3:PutBucketNotification'
Resource: !Sub 'arn:aws:s3:::${NotificationBucket}'
- Effect: Allow
Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
Resource: 'arn:aws:logs:*:*:*'
CustomResourceLambdaFunction:
Type: 'AWS::Lambda::Function'
Properties:
Handler: index.lambda_handler
Role: !GetAtt LambdaIAMRole.Arn
Code:
ZipFile: |
from __future__ import print_function
import json
import boto3
import cfnresponse
SUCCESS = "SUCCESS"
FAILED = "FAILED"
print('Loading function')
s3 = boto3.resource('s3')
def lambda_handler(event, context):
print("Received event: " + json.dumps(event, indent=2))
responseData={}
try:
if event['RequestType'] == 'Delete':
print("Request Type:",event['RequestType'])
Bucket=event['ResourceProperties']['Bucket']
delete_notification(Bucket)
print("Sending response to custom resource after Delete")
elif event['RequestType'] == 'Create' or event['RequestType'] == 'Update':
print("Request Type:",event['RequestType'])
LambdaArn=event['ResourceProperties']['LambdaArn']
Bucket=event['ResourceProperties']['Bucket']
add_notification(LambdaArn, Bucket)
responseData={'Bucket':Bucket}
print("Sending response to custom resource")
responseStatus = 'SUCCESS'
except Exception as e:
print('Failed to process:', e)
responseStatus = 'FAILURE'
responseData = {'Failure': 'Something bad happened.'}
cfnresponse.send(event, context, responseStatus, responseData)
def add_notification(LambdaArn, Bucket):
bucket_notification = s3.BucketNotification(Bucket)
response = bucket_notification.put(
NotificationConfiguration={
'LambdaFunctionConfigurations': [
{
'LambdaFunctionArn': LambdaArn,
'Events': [
's3:ObjectCreated:*'
]
}
]
}
)
print("Put request completed....")
def delete_notification(Bucket):
bucket_notification = s3.BucketNotification(Bucket)
response = bucket_notification.put(
NotificationConfiguration={}
)
print("Delete request completed....")
Runtime: python3.6
Timeout: 50
LambdaTrigger:
Type: 'Custom::LambdaTrigger'
DependsOn: LambdaInvokePermission
Properties:
ServiceToken: !GetAtt CustomResourceLambdaFunction.Arn
LambdaArn: !GetAtt S3NotificationLambdaFunction.Arn
Bucket: !Ref NotificationBucket2. To launch an AWS CloudFormation stack with the LambdaS3.template file, use the AWS CloudFormation console or the following AWS Command Line Interface (AWS CLI) command:
aws cloudformation create-stack --stack-name lambda-s3-notification --template-body file://LambdaS3.template --parameters ParameterKey=NotificationBucket,ParameterValue=existing-bucket-for-lambda-notification --capabilities CAPABILITY_NAMED_IAM --region us-east-1Important: When you launch your AWS CloudFormation stack, you must pass in your S3 bucket (existing-bucket-for-lambda-notification).
The stack creates a Lambda function and Lambda permissions for Amazon S3. Now, you can use your S3 bucket for Lambda notifications, because the stack added the required notification configuration to your S3 bucket.
Did this article help?
Do you need billing or technical support?