如何在 AWS CloudFormation 中将自定义资源与 Amazon S3 存储桶结合起来使用?

上次更新时间:2020 年 8 月 19 日

我想在 AWS CloudFormation 中将自定义资源与 Amazon Simple Storage Service (Amazon S3) 结合起来使用,以便在创建 S3 存储桶后可以执行标准操作。

简短描述

您可以使用以下解决方案中的 AWS CloudFormation 模板,将自定义资源与 AWS CloudFormation 中的 S3 存储桶结合起来使用。

请考虑以下事项:

  • 该模板支持您在 S3 存储桶中创建文件夹。Amazon S3 具有扁平式结构,但支持将文件夹概念作为对对象进行分组的一种手段。
  • 您可以使用模板在创建 S3 存储桶后执行操作,操作包括复制内容、上传内容和同步两个不同的存储桶。
  • 您可以使用自己的代码修改模板。
  • 此模板使用受 AWS Lambda 支持的自定义资源,并假定您熟悉 Lambda 最佳实践问题排查问题

注意:在以下解决方案中,删除 AWS CloudFormation 堆栈时,所有 S3 存储桶内容都会被删除。您可以通过修改 Lambda 代码来修改此行为。

解决方法

获取 AWS CloudFormation 模板

要使用 AWS CloudFormation 在 S3 存储桶中创建文件夹,请将以下 AWS CloudFormation 模板另存为 YAML 文件:

Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the "Software"), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify,
merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

AWSTemplateFormatVersion: 2010-09-09
Description: Working with custom resources and S3
Parameters:
  S3BucketName:
    Type: String
    Description: "S3 bucket to create."
    AllowedPattern: "[a-zA-Z][a-zA-Z0-9_-]*"
  DirsToCreate:
    Description: "Comma delimited list of directories to create."
    Type: CommaDelimitedList
Resources:
  SampleS3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref S3BucketName
  S3CustomResource:
    Type: Custom::S3CustomResource
    Properties:
      ServiceToken: !GetAtt AWSLambdaFunction.Arn
      the_bucket: !Ref SampleS3Bucket
      dirs_to_create: !Ref DirsToCreate
  AWSLambdaFunction:
     Type: "AWS::Lambda::Function"
     Properties:
       Description: "Work with S3 Buckets!"
       FunctionName: !Sub '${AWS::StackName}-${AWS::Region}-lambda'
       Handler: index.handler
       Role: !GetAtt AWSLambdaExecutionRole.Arn
       Timeout: 360
       Runtime: python3.6
       Code:
         ZipFile: |
          import boto3
          import cfnresponse
          def handler(event, context):
              # Init ...
              the_event = event['RequestType']
              print("The event is: ", str(the_event))
              response_data = {}
              s_3 = boto3.client('s3')
              # Retrieve parameters
              the_bucket = event['ResourceProperties']['the_bucket']
              dirs_to_create = event['ResourceProperties']['dirs_to_create']
              try:
                  if the_event in ('Create', 'Update'):
                      print("Requested folders: ", str(dirs_to_create))
                      for dir_name in dirs_to_create:
                          print("Creating: ", str(dir_name))
                          s_3.put_object(Bucket=the_bucket,
                                         Key=(dir_name
                                              + '/'))
                  elif the_event == 'Delete':
                      print("Deleting S3 content...")
                      b_operator = boto3.resource('s3')
                      b_operator.Bucket(str(the_bucket)).objects.all().delete()
                  # Everything OK... send the signal back
                  print("Execution succesfull!")
                  cfnresponse.send(event,
                                   context,
                                   cfnresponse.SUCCESS,
                                   response_data)
              except Exception as e:
                  print("Execution failed...")
                  print(str(e))
                  response_data['Data'] = str(e)
                  cfnresponse.send(event,
                                   context,
                                   cfnresponse.FAILED,
                                   response_data)
  AWSLambdaExecutionRole:
     Type: AWS::IAM::Role
     Properties:
       AssumeRolePolicyDocument:
         Statement:
         - Action:
           - sts:AssumeRole
           Effect: Allow
           Principal:
             Service:
             - lambda.amazonaws.com
         Version: '2012-10-17'
       Path: "/"
       Policies:
       - PolicyDocument:
           Statement:
           - Action:
             - logs:CreateLogGroup
             - logs:CreateLogStream
             - logs:PutLogEvents
             Effect: Allow
             Resource: arn:aws:logs:*:*:*
           Version: '2012-10-17'
         PolicyName: !Sub ${AWS::StackName}-${AWS::Region}-AWSLambda-CW
       - PolicyDocument:
           Statement:
           - Action:
             - s3:PutObject
             - s3:DeleteObject
             - s3:List*
             Effect: Allow
             Resource:
             - !Sub arn:aws:s3:::${SampleS3Bucket}/*
             - !Sub arn:aws:s3:::${SampleS3Bucket}
           Version: '2012-10-17'
         PolicyName: !Sub ${AWS::StackName}-${AWS::Region}-AWSLambda-S3
       RoleName: !Sub ${AWS::StackName}-${AWS::Region}-AWSLambdaExecutionRole

部署 AWS CloudFormation 模板

您可以使用 AWS CloudFormation 控制台或 AWS 命令行界面 (AWS CLI) 来部署 AWS CloudFormation 模板。

使用 AWS CloudFormation 控制台:

1.    打开 AWS CloudFormation 控制台

2.    选择创建堆栈,然后选择使用新资源(标准)

3.    在指定模板部分,选择上传模板文件

4.    选择选择文件,选择您在步骤 1 中下载的模板,然后选择下一步

5.    在参数部分,对于 S3BucketName,选择您的 S3 存储桶。

6.    对于 DirsToCreate,输入想要创建的文件夹和子文件夹列表,文件夹之间使用逗号隔开。

注意:例如,列表您可以输入 dir_1,dir_2/sub_dir_2,dir_3

7.    完成安装向导中的其余步骤,然后选择创建堆栈

使用 AWS CLI:

1.    命名您下载的模板 custom-resource-lmabda-s3.yaml

2.    安装 AWS CLI

3.    在操作系统中打开命令行,然后转到模板所在的文件夹。

4.    运行以下命令:

aws cloudformation create-stack \
                   --timeout-in-minutes 10 \
                   --disable-rollback \
                   --stack-name testing-custom-resource-s3 \
                   --template-body file://custom-resource-lmabda-s3.yaml \
                   --capabilities CAPABILITY_NAMED_IAM \
                   --parameters \
                   ParameterKey=DirsToCreate,ParameterValue="dir_1\,dir_2/sub_dir_2\,dir_3" \
                   ParameterKey=S3BucketName,ParameterValue="test-bucket-custom-resource"

节点:参数更新为您的值。


这篇文章对您有帮助吗?


您是否需要账单或技术支持?