Automating Amazon S3 on Outposts resources with AWS CloudFormation

We recently announced support for AWS CloudFormation templates to create and manage Amazon S3 on Outposts resources, like buckets, endpoints, and access points. Until recently, customers were limited to creation of such resources individually using the AWS Management Console, AWS CLI, or SDKs. Now, by leveraging AWS CloudFormation, you can automate creation of Amazon S3 on Outposts buckets, access points, endpoints, and policies. Customers can now benefit from the familiarity of CloudFormation and have the ability to control a resource throughout its deployment lifecycle. This simplifies the customer experience of deploying and managing S3 resources on Outposts in a consistent and reliable manner, which is increasingly important as the number of customer use cases spanning multiple resources supported on Outposts grows.

In this post, to explore this new functionality, we walk through using CloudFormation to automate creation of key S3 on Outposts resources. This walk though will enable you to create a S3 on Outposts endpoint, a bucket on your Outpost, and an access point to manage access to your bucket. S3 on Outposts provides object-storage on-premises to customers with Outposts applications that use local Amazon S3 storage, have local data processing needs, and have data residency requirements. AWS CloudFormation is an infrastructure as code tool that gives you the ability to quickly and consistently deploy and manage your resources.

Prerequisites

Before we begin creating a bucket and its Access Point, let us discuss our current setup. We have an AWS account with an on-premises Outpost anchored to the us-west-2 Region. I have also already deployed the following resources with another CloudFormation stack, which I refer to as my network stack:

• Amazon Virtual Private Cloud (Amazon VPC)
• Outpost subnet
• Security group

If you need additional guidance on creating the preceding resources, please refer to the CloudFormation resource documentation.

I am also working with an Outpost on which Amazon S3 on Outposts capacity is already enabled. To learn how to get S3 deployed on your Outposts, refer to the S3 on Outposts documentation.

Amazon S3 on Outposts resource creation

Now let us begin, with the end goal of creating a bucket that will be available to store data in.

1. The first thing we recommend that customers do is to create their Amazon S3 on Outposts endpoint. S3 on Outposts uses endpoints to connect to S3 on Outposts buckets so that you can perform actions within your VPC.

We have added the following code to my network stack and updated the stack.

Amazon S3 on Outposts endpoint snippet:

Resources:
   S3OnOutpostsEndpoint:
     Type: AWS::S3Outposts::Endpoint
     Properties:
       OutpostId: !Ref OutpostID
       SecurityGroupId: !Ref mySG
       SubnetId: !Ref OutpostSubnet

2. Now, we are ready to create our bucket and Access Point. I have these two items paired together because Amazon S3 on Outposts buckets require an access point. You use this access point to access the bucket to perform object operations. For more information about access points, please see the documentation on accessing Amazon S3 on Outposts.

AWSTemplateFormatVersion: '2010-09-09'
Description: An S3 on Outposts Bucket
Parameters:
  BucketName:
    Type: String
    Default: 'outpostsbucket'
  OutpostID:
    Type: String
    Default: 'op-0268f76782a30c66a'
   VPCID:
    Type: String
    Default: 'vpc-09b7e36e295b08df8'
  AccessPointName:
    Type: String
    Default: 'outpostaccesspoint'
Resources:
  S3OnOutpostsBucket:
    Type: AWS::S3Outposts::Bucket
    Properties:
      BucketName: !Ref BucketName
      OutpostId: !Ref OutpostID
  S3OnOutpostsAccessPoint:
    Type: AWS::S3Outposts::AccessPoint
    Properties:
      Bucket: !Ref S3OnOutpostsBucket
      Name: !Ref AccessPointName
      VpcConfiguration:
        VpcId: !Ref VPCID
      Policy:
        Version: '2012-10-17'
        Id: AccessPointPolicy
        Statement:
        - Sid: st1
          Effect: Allow
          Action: "s3-outposts:*"
          Principal:
            AWS:
              - !Join ["", [ "arn:aws:iam::", !Ref 'AWS::AccountId', ":root" ] ]
          Action: s3-outposts:*
          Resource: !Join
            - ""
            - - "arn:"
              - !Ref 'AWS::Partition'
              - ":s3-outposts:"
              - !Ref 'AWS::Region'
              - ":"
              - !Ref 'AWS::AccountId'
              - ":outpost/"
              - !Ref 'OutpostID'
              - "/accesspoint/"
              - !Ref 'AccessPointName'

Outputs:
  OutpostsBucketARN:
    Description: The ARN of OutpostsBucket
    Value:
      Ref: S3OnOutpostsBucket
  OutpostsAccessPointARN:
    Description: The ARN of OutpostsAccessPoint
    Value:
      Ref: S3OnOutpostsAccessPoint
  OutpostsBucketStackId:
    Description: The Stack ID of OutpostsBucket
    Value:
      Ref: AWS::StackId
    Export:
      Name:
        Fn::Sub: "${AWS::StackName}-StackId"

The preceding template should be a familiar AWS CloudFormation template. If this is not familiar, please look at the documentation on CloudFormation concepts.

In this template, you will see a new resource AWS::S3Outposts::Bucket and AWS::S3Outposts::AccessPoint. If you plan to use existing CloudFormation stacks with Amazon S3 on Outpost, ensure you update your template to use these new resources. After applying the CloudFormation stack, you can see that it has been created successfully and view stack details in the Stack info tab.

After applying this stack, if I browse to the Amazon S3 on Outposts resources in the AWS Management Console, I see my bucket.

You now have an S3 on Outposts bucket, access point, and endpoint configured and are ready to start using them. This covers the basic set of resources to get started with S3 on Outposts. Next, let’s look at adding an Access Point policy so that we can restrict access to a certain IAM principal. If we look at the code we have already run, it allows communication from all IAM users and roles in my account. However, if you have a situation where you want to restrict this communication from a certain IAM principal you could refactor this code like this. Note that I have added another parameter called RoleArn to avoid hardcoded values in my code.

S3OnOutpostsAccessPoint:
    Type: AWS::S3Outposts::AccessPoint
    Properties:
      Bucket: !Ref S3OnOutpostsBucket
      Name: !Ref AccessPointName
      VpcConfiguration:
        VpcId: !Ref VPCID
      Policy:
        Version: '2012-10-17'
        Id: AccessPointPolicy
        Statement:
        - Sid: st1
          Effect: Allow
          Action: "s3-outposts:*"
          Principal:
            AWS: !Ref RoleArn
          Action: s3-outposts:*       
          Resource: !Join
            - ""
            - - "arn:"
              - !Ref 'AWS::Partition'
              - ":s3-outposts:"
              - !Ref 'AWS::Region'
              - ":"
              - !Ref 'AWS::AccountId'
              - ":outpost/"
              - !Ref 'OutpostID'
              - "/accesspoint/"
              - !Ref 'AccessPointName'

After some time, you may find yourself consuming a great deal of Amazon S3 storage, and you may want to reclaim some of this space. If you’re in this position, be aware that the AWS::S3Outposts::Bucket resource supports S3 Lifecycle configuration for age-based expiration of objects.

You can control the S3 Lifecycle configuration a few different ways. For a list of possible rules, see the documentation on AWS::S3Outposts::Bucket Rule.

S3OnOutpostsBucket:
    Type: AWS::S3Outposts::Bucket
    Properties:
      BucketName: !Ref BucketName
      OutpostId: !Ref OutpostID
      LifecycleConfiguration: 
        Rules:
          - ExpirationInDays: 100
	    Status: Enabled

Summary

You now are enabled to use CloudFormation and have the ability to bring Amazon S3 on Outposts resource creation and management into your existing, and new, CloudFormation stacks.

Thanks for reading this blog post! If you have any comments or questions, please leave them in the comments section. Happy coding and looking forward to hearing about what you build on Outposts.