AWS re:Invent recap: Extend Amazon S3 to on-premises environments with AWS Outposts

While we could not meet with you in person at re:Invent this year, we hope that you enjoyed joining us for our first ever virtual re:Invent – enabling you to get the full re:Invent experience from home, and for free.

In this blog post, I highlight Amazon S3 on Outposts, launched on September 30, 2020 to provide Amazon S3 object storage in your on-premises environments. This launch and the benefits it provides was covered in the re:Invent session, “Extend Amazon S3 to on-premises environments with AWS Outposts,” which is now available on-demand, and which I recap in this post. In the re:Invent session, my colleague Paul Reed covers an introduction to the service and use case patterns we are seeing emerge across customers. In addition, in the session Paul deep dives on key features and developer experiences.

First, why AWS Outposts?

As you’ve heard across re:Invent sessions, AWS has a large global footprint spanning 77 Availability Zones and 220 points of presence across 24 geographic Regions globally. We’ve already announced plans for 6 more AWS Regions and 15 more Availability Zones, and introduced plans for new form factors to bring AWS closer to your end users through 12 new Local Zones.

Despite this broad range of platforms, we hear from customers who want to move to the cloud that some applications are harder than others to migrate. On one end of the spectrum, there are web-based applications that tend to be easier to bring into the cloud. On the other end of the spectrum, we hear from customers building applications that must process large amounts of data generated locally. Some are subject to the laws of physics in terms of latency sensitive responses in contexts such as robotics and manufacturing automation. There are also customers looking to migrate legacy workloads with complex dependencies on on-premises infrastructure, who are looking for ways to incrementally evolve architectures to reduce risk to their organizations.

A common theme across these customers is that they want the same experience, tools, and monitoring capabilities available to them in the cloud, accessible across all their environments. In response to these customers, AWS launched AWS Outposts in 2019, bringing the AWS Cloud into customer data centers for a truly consistent hybrid cloud experience. With the launch of Amazon S3 on Outposts, AWS now provides a comprehensive, single-vendor compute and storage solution that is designed to meet the needs of customers who want to use modern, cloud compatible architectures to run workloads locally. Once installed, we take care of monitoring, maintaining, and upgrading your Outposts. Outposts hardware features standard hardware we use in our own AWS data centers, with some additional security devices. Outposts racks are standard 42U racks that arrive fully assembled – we roll them in on casters, ready for connection to power and networking.

Everything that you and your team already know about AWS still applies. You use the same APIs, tools, and operational practices. You can create a single deployment pipeline that targets your AWS Outposts and your cloud environments, and you can create hybrid architectures that span both. Since launching Outposts in 2019 with Amazon EC2 and Amazon EBS, we have added support for additional AWS services such as Application Load Balancer, Amazon ElastiCache, and Amazon Relational Database Service on Outposts over the past year.

Amazon S3 on AWS Outposts

In response to one the most frequent asks to enhance storage options on AWS Outposts, AWS launched Amazon S3 on Outposts in September 2020. S3 on Outposts is designed for customers with Outposts applications that use local Amazon S3 storage, have local data processing needs, and have data residency requirements. S3 on Outposts is available as a new S3 storage class, designed to durably store objects across multiple devices and servers. S3 on Outposts provides the same S3 APIs and a set of familiar features, with some differences to accommodate operation on your Outpost as compared to AWS Regions. You access your data through an endpoint on your Outposts ensuring that data remains both stored and transferred within facilities and networks that you control. The same AWS Identity and Access Management (IAM) mechanisms available in AWS Regions today can be used to secure and control access to objects stored on Outposts, enabling consistent controls across all your data. You can use features such as object and bucket tagging to simplify management and access control, and you can use S3 Lifecycle expiration to automatically manage data retention policies.

Use cases

Since launching the capability, we have observed a few patterns in how customers are leveraging the service to enhance applications built on Outposts.

Data residency

A primary set of use cases for Amazon S3 on Outposts come from customers looking to adhere to local regulations that dictate where data must be stored. Often, data may be stored either within a specific location, or within certain geographic boundaries for compliance reasons. This includes healthcare and life sciences customers who may need to store patient data and records locally, or organizations servicing government entities that must store data within particular geographies.

Local processing

Customers in industries such as robotics and autonomous vehicles have contexts that generate large amounts of data locally. S3 on Outposts allows these customers to store and filter data local and only transfer critical data back to AWS Regions for analysis or archival. This minimizes data transfers and reduces the impact of network latencies and bandwidth constraints on the end-to-end workflow.

Staging and migration

Customers are looking to use Outposts as an intermediate step in the cloud migration journey. Customers with workloads running in existing data centers want to migrate these applications to the cloud for increased agility. Many times, they find that moving legacy applications is not easy due to dependencies on local infrastructure and other resources on premises. Outposts installations on premises allow these customers to take a more incremental approach to get to the cloud by providing access to the cloud programming model on premises.

A consistent Amazon S3 experience

Amazon S3 on Outposts uses the same S3 APIs to store and retrieve objects as you use today: your GET, PUT, LIST, and HEAD operations are consistent regardless of whether you are accessing objects in buckets stored on Outposts or in an AWS Region. Data stored in S3 on Outposts is stored locally on your Outpost. We expanded the S3 control API to enable creation and management of buckets on specific Outposts by specifying the ID of your Outposts in place of a Region identifier. This means that while you manage bucket management and creation through yours Outpost’s home Region, all objects and object metadata are stored locally on your Outpost. We’ve also added a new SDK and commands to the AWS CLI for management and routing to endpoint resources specific to S3 on Outposts.

AWS offers two capacity options for S3 on Outposts today: a 48-TB option, and a 96-TB option. Customers have access to Amazon CloudWatch metrics to monitor capacity utilization and bucket sizes on Outposts. In addition, AWS CloudTrail logs are available for both management and data events related to your S3 on Outposts buckets. Data stored on your Outposts is always encrypted using server-side encryption with service managed keys (SSE-S3). You also elect to use server-side encryption with your own keys (SSE-C).

In addition, AWS DataSync now supports S3 on Outposts as a data transfer location, allowing automation of data transfers between Outposts to and from AWS Regions.

Conclusion

In this session, we introduced Amazon S3 on Outposts through a few examples of patterns in which we see our customers using AWS Outposts. In addition, we covered an overview of how the capability works on Outposts, along with providing an example workflow of setting up a bucket, access controls, and a simple PUT/ GET to a bucket stored on Outposts.

To get started with Amazon S3 on Outposts, visit the AWS Outposts pricing page to select a configuration that is suited to your application needs. S3 capacity can be added to most Outposts configurations. Once we deliver and install your Outpost, you are able to create buckets, and store and retrieve data into these buckets as you do in the Region today.

Thank you again for joining us at re:Invent this year! We at AWS always appreciate the opportunity to share more about our services and hear feedback from our customers through the event. We particularly appreciate the opportunity to share product updates and examples of how we see customers designing solutions with new services such as Amazon S3 on Outposts.

We look forward to hearing about what you build with your Outposts and S3 on Outposts! If you have any comments or questions regarding the launch or the re:Invent session, please don’t hesitate to leave them in the comments section.