How do I avoid the error "Unable to validate the following destination configurations" when using S3 event notifications in AWS CloudFormation?

Last updated: 2019-11-08

I subscribe an Amazon Simple Notification Service (Amazon SNS) topic or an AWS Lambda function to Amazon Simple Storage Service (Amazon S3) event notifications. Then, I receive the following error: "Unable to validate the following destination configurations." When I try to set a dependency on the SNS topic policy from the S3 bucket in my template, I receive a circular dependency validation error.

Short Description

Because of the way AWS CloudFormation handles dependency ordering, Amazon S3 event notifications are defined as an attribute of the S3 bucket. These notifications are established when the S3 bucket resource is created.

To avoid an error, you must create resources in the following order:

  1. Create the SNS topic, because the S3 bucket references the SNS topic.
  2. Create the S3 bucket, because the SNS topic policy references both the S3 bucket and the SNS topic.

Before subscribing an SNS topic to S3 event notifications, you must specify a topic policy (AWS::SNS::TopicPolicy) with the appropriate permissions. That topic policy must exist before you create the subscription.

Resolution

Try one of the following strategies to avoid the "Unable to validate the following destination configurations" error:

Specify a value for BucketName in your AWS CloudFormation template

Use a static name for your S3 bucket by specifying a value for the BucketName property in the S3Bucket resource of your AWS CloudFormation template. This removes the need to include {"Ref": "paramBucketName"} in the SNS topic policy, and removes the intrinsic dependency between the SNS topic policy and Amazon S3.

The following example AWS CloudFormation template specifies a hardcoded value (-Bucket-Name-) for the BucketName property. In your template, replace all instances of -Bucket-Name- with your bucket name.

{
  "Resources": {
    "SNSTopic": {
        "Type": "AWS::SNS::Topic"
    },
    "SNSTopicPolicy": {
        "Type": "AWS::SNS::TopicPolicy",
        "Properties": {
            "PolicyDocument": {
                "Id": "MyTopicPolicy",
                "Version": "2012-10-17",
                "Statement": [
                    {
                        "Sid": "Statement-id",
                        "Effect": "Allow",
                        "Principal": { "AWS": "*" },
                        "Action": "sns:Publish",
                        "Resource": { "Ref": "SNSTopic" },
                        "Condition": {
                            "ArnLike": {
                                "aws:SourceArn": { "Fn::Join": [ "", [ "arn:aws:s3:::", "-Bucket-Name-" ]]} }
                        }
                    }
                ]
            },
            "Topics": [ { "Ref": "SNSTopic" } ]
        }
    },
    "S3Bucket": {
        "Type": "AWS::S3::Bucket",
        "DependsOn": ["SNSTopicPolicy"],
        "Properties": {
            "AccessControl": "BucketOwnerFullControl",
            "BucketName": "-Bucket-Name-",
            "NotificationConfiguration": {
                "TopicConfigurations": [
                    {
                        "Topic": { "Ref": "SNSTopic" },
                        "Event": "s3:ObjectCreated:Put"
                    }
                ]
            }
        }
    }
  }
}

Note: The S3Bucket resource has an explicit DependsOn value that's set to SNSTopicPolicy. This attribute specifies that the template will create the SNSTopicPolicy resource before the S3Bucket resource.

To use the same AWS CloudFormation template for S3 buckets with different names, set up a parameter for the bucket name. The parameter allows you to use the same template for different bucket names by passing the bucket name as a parameter during stack creation.

To use the following example template, you must pass the bucket name as the paramBucketName parameter during stack creation.

{
  "Parameters": {
    "paramBucketName": {
      "Type": "String",
      "Description": "Bucket Name"
    }
  },
  "Resources": {
    "SNSTopic": {
        "Type": "AWS::SNS::Topic"
    },
    "SNSTopicPolicy": {
        "Type": "AWS::SNS::TopicPolicy",
        "Properties": {
            "PolicyDocument": {
                "Id": "MyTopicPolicy",
                "Version": "2012-10-17",
                "Statement": [
                    {
                        "Sid": "Statement-id",
                        "Effect": "Allow",
                        "Principal": { "AWS": "*" },
                        "Action": "sns:Publish",
                        "Resource": { "Ref": "SNSTopic" },
                        "Condition": {
                            "ArnLike": {
                                "aws:SourceArn": { "Fn::Join": [ "", [ "arn:aws:s3:::", {"Ref": "paramBucketName"} ]]} }
                        }
                    }
                ]
            },
            "Topics": [ { "Ref": "SNSTopic" } ]
        }
    },
    "S3Bucket": {
        "Type": "AWS::S3::Bucket",
        "DependsOn": ["SNSTopicPolicy"],
        "Properties": {
            "AccessControl": "BucketOwnerFullControl",
            "BucketName": {"Ref": "paramBucketName"},
            "NotificationConfiguration": {
                "TopicConfigurations": [
                    {
                        "Topic": { "Ref": "SNSTopic" },
                        "Event": "s3:ObjectCreated:Put"
                    }
                ]
            }
        }
    }
  }
}

Create a stack, and then perform a stack update

Divide the stack creation into two stages. First, create the stack, but don't specify the NotificationConfiguration property in the S3Bucket resource. Then, perform a stack update to add the S3 event notification. This avoids setting the S3 event notification before the SNS topic policy is created.

1.    Create all the resources, including the SNS topic policy. See the following example:

{
    "Resources": {
        "SNSTopic": {
            "Type": "AWS::SNS::Topic"
        },
        "SNSTopicPolicy": {
            "Type": "AWS::SNS::TopicPolicy",
            "Properties": {
                "PolicyDocument": {
                    "Id": "MyTopicPolicy",
                    "Version": "2012-10-17",
                    "Statement": [
                        {
                            "Sid": "Statement-id",
                            "Effect": "Allow",
                            "Principal": { "AWS": "*" },
                            "Action": "sns:Publish",
                            "Resource": { "Ref": "SNSTopic" },
                            "Condition": {
                                "ArnLike": {
                                    "aws:SourceArn": { "Fn::Join": [ "", [ "arn:aws:s3:::", { "Ref": "S3Bucket" } ] ]
                                    }
                                }
                            }
                        }
                    ]
                },
                "Topics": [ { "Ref": "SNSTopic" } ]
            }
        },
        "S3Bucket": {
            "Type": "AWS::S3::Bucket",
            "Properties": {
                "AccessControl": "BucketOwnerFullControl"
            }
        }
    }
}

2.    Update the stack to add the S3 event notification. See the following example:

{
  "Resources": {
        "SNSTopic": {
        "Type": "AWS::SNS::Topic"
    },
    "SNSTopicPolicy": {
        "Type": "AWS::SNS::TopicPolicy",
        "Properties": {
            "PolicyDocument": {
                "Id": "MyTopicPolicy",
                "Version": "2012-10-17",
                "Statement": [
                    {
                        "Sid": "Statement-id",
                        "Effect": "Allow",
                        "Principal": { "AWS": "*" },
                        "Action": "sns:Publish",
                        "Resource": { "Ref": "SNSTopic" },
                        "Condition": {
                            "ArnLike": {
                                "aws:SourceArn": { "Fn::Join": [ "", [ "arn:aws:s3:::", { "Ref": "S3Bucket" } ] ]
                                }
                            }
                        }
                    }
                ]
            },
            "Topics": [ { "Ref": "SNSTopic" } ]
        }
    },
    "S3Bucket": {
        "Type": "AWS::S3::Bucket",
        "Properties": {
            "AccessControl": "BucketOwnerFullControl",
            "NotificationConfiguration": {
                "TopicConfigurations": [
                    {
                        "Topic": { "Ref": "SNSTopic" },
                        "Event": "s3:ObjectCreated:Put"
                    }
                ]
            }
        }
    }
  }
}

Note: You can convert your AWS CloudFormation templates between JSON and YAML formats.