How do I avoid the error "Unable to validate the following destination configurations" when using Lambda event notifications in AWS CloudFormation?

Last updated: 2019-09-25

My stack fails when I deploy an AWS CloudFormation template, and I receive an error similar to the following: "Unable to validate the following destination configurations."

Short Description

You receive this error when you deploy an AWS CloudFormation template with the following resources:

When creating the bucket, Amazon S3 must validate the notification configuration by checking if the bucket has permission to push events to the Lambda function. The permission resource (which must exist for this check to pass) requires the bucket name. This means the permission resource depends on the bucket, and the bucket depends on the permission resource.

Note: You receive the "Circular dependency between resources" error if you try to resolve this issue by implementing a DependsOn resource attribute similar to the following code example.

The following code example shows a circular dependency between the S3 bucket resource and the SourceArn property of the Lambda permission resource.

"MyS3BucketPermission": {
  "Properties": {
    "Action": "lambda:InvokeFunction",
    ...
    ...
    "SourceArn": {
      "Ref": "MyS3Bucket"
    }
  },
  "Type": "AWS::Lambda::Permission"
},
"Resources": {
  "MyS3Bucket": {
    "DependsOn" : "MyS3BucketPermission",

Note: It's a best practice to add the SourceAccount property to the Lambda permission resource for Amazon S3 event sources, because an Amazon Resource Name (ARN) for Amazon S3 doesn't include an account ID. The SourceArn property is adequate for most other event sources, but consider adding the SourceAccount property for Amazon S3 event sources. This prevents a scenario where you delete the S3 bucket only to find that someone else has re-created the bucket, granting the new owner of the bucket full permissions to invoke your Lambda function.

Resolution

You can avoid circular dependencies by using the Fn::Join intrinsic function with stack parameters.

In the following sample template, the S3 bucket name BucketPrefix is provided as a parameter to AWS::S3::Bucket and AWS::Lambda::Permission resources.

Note: The following example assumes that the bucket name wasn't used previously with your AWS accounts. If you want to reuse a template with this code snippet, you must provide a different bucket prefix every time.

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Parameters": {
    "BucketPrefix": {
      "Type": "String",
      "Default": "test-bucket-name"
    }
  },
  "Resources": {
    "EncryptionServiceBucket": {
      "DependsOn": "LambdaInvokePermission",
      "Type": "AWS::S3::Bucket",
      "Properties": {
        "BucketName": {
          "Fn::Sub": "${BucketPrefix}-encryption-service"
        },
        "NotificationConfiguration": {
          "LambdaConfigurations": [
            {
              "Function": {
                "Fn::GetAtt": [
                  "AppendItemToListFunction",
                  "Arn"
                ]
              },
              "Event": "s3:ObjectCreated:*",
              "Filter": {
                "S3Key": {
                  "Rules": [
                    {
                      "Name": "suffix",
                      "Value": "zip"
                    }
                  ]
                }
              }
            }
          ]
        }
      }
    },
    "LambdaInvokePermission": {
      "Type": "AWS::Lambda::Permission",
      "Properties": {
        "FunctionName": {
          "Fn::GetAtt": [
            "AppendItemToListFunction",
            "Arn"
          ]
        },
        "Action": "lambda:InvokeFunction",
        "Principal": "s3.amazonaws.com",
        "SourceAccount": {
          "Ref": "AWS::AccountId"
        },
        "SourceArn": { 
            "Fn::Sub": "arn:aws:s3:::${BucketPrefix}-encryption-service"
        },
    "AppendItemToListFunction": {
      "Type": "AWS::Lambda::Function",
      "Properties": {
        "Handler": "index.handler",
        "Role": {
          "Fn::GetAtt": [
            "LambdaExecutionRole",
            "Arn"
          ]
        },
        "Code": {
          "ZipFile": {
            "Fn::Join": [
              "",
              [
                "var response = require('cfn-response');",
                "exports.handler = function(event, context) {",
                " var responseData = {Value: event.ResourceProperties.List};",
                " responseData.Value.push(event.ResourceProperties.AppendedItem);",
                " response.send(event, context, response.SUCCESS, responseData);",
                "};"
              ]
            ]
          }
        },
        "Runtime": "nodejs8.10"
      }
    },
    "LambdaExecutionRole": {
      "Type": "AWS::IAM::Role",
      "Properties": {
        "AssumeRolePolicyDocument": {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Effect": "Allow",
              "Principal": {
                "Service": [
                  "lambda.amazonaws.com"
                ]
              },
              "Action": [
                "sts:AssumeRole"
              ]
            }
          ]
        },
        "Path": "/",
        "Policies": [
          {
            "PolicyName": "root",
            "PolicyDocument": {
              "Version": "2012-10-17",
              "Statement": [
                {
                  "Effect": "Allow",
                  "Action": [
                    "logs:*"
                  ],
                  "Resource": "arn:aws:logs:*:*:*"
                }
              ]
            }
          }
        ]
      }
    }
  }
}

The template avoids circular dependency because it creates the resources in the following order:

  1. AWS Identity and Access Management (IAM) role
  2. Lambda function
  3. Lambda permission
  4. S3 bucket

Now, Amazon S3 can verify its notification configuration and create the bucket without any issues.

You can also try the following workarounds:

  • Create the S3 bucket without notification configuration, and then add the bucket in the next stack update.
  • Create a less-constrained Lambda permission. For example, allow invocations for a specific AWS account by omitting SourceArn.
  • Create a custom resource to run at the end of the stack workflow. This resource adds the notification configuration to the S3 bucket after all other resources are created.