How do I avoid the "Unable to validate the following destination configurations" error with Lambda event notifications in CloudFormation?

4 minute read
0

My stack fails when I deploy an AWS CloudFormation template. Then, I receive an error similar to the following: "Unable to validate the following destination configurations."

Short description

You receive this error when you deploy a CloudFormation template with the following resources:

Amazon S3 must validate the notification configuration when it creates the bucket. The validation is done by checking if the bucket has permission to push events to the Lambda function. The permission resource (which must exist for this check to pass) requires the bucket name. This means that the permission resource depends on the bucket, and the bucket depends on the permission resource.

Note: You receive the "Circular dependency between resources" error if you try to resolve this issue by implementing a DependsOn resource attribute similar to the following code example.

The following example shows a circular dependency between the S3 bucket resource and the SourceArn property of the Lambda permission resource.

"Resources": {
  "MyS3BucketPermission": {
    "Type": "AWS::Lambda::Permission",
    "Properties": {
      "Action": "lambda:InvokeFunction",
      ...
      ...
      "SourceArn": {
        "Ref": "MyS3Bucket"
      }
    }
  },
  "MyS3Bucket": {
    "DependsOn" : "MyS3BucketPermission",
    ...
    ...

Important: It's a best practice to add the SourceAccount property to the Lambda permission resource for Amazon S3 event sources. You add the property because an Amazon Resource Name (ARN) for Amazon S3 doesn't include an account ID. The SourceArn property is adequate for most other event sources, but consider adding the SourceAccount property for Amazon S3 event sources. This prevents users from re-creating a bucket that you deleted, and then granting a new bucket owner full permissions to invoke your Lambda function.

Resolution

You can avoid circular dependencies by using the Fn::Sub intrinsic function with stack parameters. You can also use Fn::Join to combine strings.

In the following sample template, the S3 bucket name BucketPrefix is a parameter for AWS::S3::Bucket and AWS::Lambda::Permission resources.

Note: The following example assumes that the bucket name wasn't used previously with your AWS accounts. If you want to reuse a template with this code snippet, then you must provide a different bucket prefix every time.

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Parameters": {
    "BucketPrefix": {
      "Type": "String",
      "Default": "test-bucket-name"
    }
  },
  "Resources": {
    "EncryptionServiceBucket": {
      "DependsOn": "LambdaInvokePermission",
      "Type": "AWS::S3::Bucket",
      "Properties": {
        "BucketName": {
          "Fn::Sub": "${BucketPrefix}-encryption-service"
        },
        "NotificationConfiguration": {
          "LambdaConfigurations": [
            {
              "Function": {
                "Fn::GetAtt": [
                  "AppendItemToListFunction",
                  "Arn"
                ]
              },
              "Event": "s3:ObjectCreated:*",
              "Filter": {
                "S3Key": {
                  "Rules": [
                    {
                      "Name": "suffix",
                      "Value": "zip"
                    }
                  ]
                }
              }
            }
          ]
        }
      }
    },
    "LambdaInvokePermission": {
      "Type": "AWS::Lambda::Permission",
      "Properties": {
        "FunctionName": {
          "Fn::GetAtt": [
            "AppendItemToListFunction",
            "Arn"
          ]
        },
        "Action": "lambda:InvokeFunction",
        "Principal": "s3.amazonaws.com",
        "SourceAccount": {
          "Ref": "AWS::AccountId"
        },
        "SourceArn": { 
            "Fn::Sub": "arn:aws:s3:::${BucketPrefix}-encryption-service"
        }
      }
    },
    "AppendItemToListFunction": {
      "Type": "AWS::Lambda::Function",
      "Properties": {
        "Handler": "index.handler",
        "Role": {
          "Fn::GetAtt": [
            "LambdaExecutionRole",
            "Arn"
          ]
        },
        "Code": {
          "ZipFile": {
            "Fn::Join": [
              "",
              [
                "var response = require('cfn-response');",
                "exports.handler = function(event, context) {",
                " var responseData = {Value: event.ResourceProperties.List};",
                " responseData.Value.push(event.ResourceProperties.AppendedItem);",
                " response.send(event, context, response.SUCCESS, responseData);",
                "};"
              ]
            ]
          }
        },
        "Runtime": "nodejs12.x"
      }
    },
    "LambdaExecutionRole": {
      "Type": "AWS::IAM::Role",
      "Properties": {
        "AssumeRolePolicyDocument": {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Effect": "Allow",
              "Principal": {
                "Service": [
                  "lambda.amazonaws.com"
                ]
              },
              "Action": [
                "sts:AssumeRole"
              ]
            }
          ]
        },
        "Path": "/",
        "Policies": [
          {
            "PolicyName": "root",
            "PolicyDocument": {
              "Version": "2012-10-17",
              "Statement": [
                {
                  "Effect": "Allow",
                  "Action": [
                    "logs:*"
                  ],
                  "Resource": "arn:aws:logs:*:*:*"
                }
              ]
            }
          }
        ]
      }
    }
  }
}

The template avoids a circular dependency because it creates the resources in the following order:

  1. AWS Identity and Access Management (IAM) role
  2. Lambda function
  3. Lambda permission
  4. S3 bucket

Now, Amazon S3 can verify its notification configuration and create the bucket without any issues.

You can also try the following resolutions:

  • Create the S3 bucket without a notification configuration, and then add the bucket in the next stack update.
  • Create a less-constrained Lambda permission. For example, allow invocations for a specific AWS account by omitting SourceArn.
  • Create a custom resource to run at the end of the stack workflow. This resource adds the notification configuration to the S3 bucket after all other resources are created.

Related information

How do I avoid the "Unable to validate the following destination configurations" error in AWS CloudFormation?

Using AWS Lambda with Amazon S3 Events

Ref

Fn::GetAtt

AWS OFFICIAL
AWS OFFICIALUpdated 3 years ago
No comments

Relevant content