AWS Storage Gateway in 2021 – Year in Review

It has become customary for us to share with our customers the new AWS Storage Gateway enhancements every year. As we did in 2020 and 2019, through this blog I’m reviewing all the new Storage Gateway launches from 2021. I’ll cover how the new enhancements make it easier for customers to access AWS Storage from applications running in environments external to AWS.

About Storage Gateway

Storage Gateway is a hybrid cloud storage service that enables on-premises workloads to use AWS Storage. You use Storage Gateway to bridge your data center, remote office, and edge applications with AWS, so you can provide local access to virtually unlimited cloud storage. The service supports standard storage protocols such as NFS, SMB, iSCSI, and iSCSI-VTL, so existing on-premises applications can use AWS Storage without any modifications to existing workflows.

Customers deploy gateways in on-premises environments adjacent to applications to deliver low latencies for frequently accessed data and optimize data transfers to AWS.  Customers manage the gateways using the AWS Management Console, API, and CLI, in the same way they manage other AWS services. Storage Gateway integrates with AWS services such as AWS Identity and Access Management (IAM), Amazon CloudWatch, and AWS CloudTrail, enabling secure access to AWS services, easy management and monitoring, and tracking of user activity on AWS resources. The picture below shows the end-to-end Storage Gateway architecture.

AWS Storage Gateway enhancements in 2021

In 2021, we delivered enhancements to address storage needs for customers’ file and tape-based applications, enhanced the user experience, and expanded availability and compliance support.

Addressed many hybrid cloud file storage needs

As customers continue their cloud journeys, they have file-based workloads that need to run on premises closer to their end users. These workloads need the agility, elasticity, and cost saving benefits of the cloud. To address our customers’ storage needs for file workloads, we launched a new gateway Amazon FSx File Gateway and added security and compliance enhancements for S3 File Gateway.

Storage Gateway launched S3 File Gateway in 2016 to enable customers to back up their on-premises databases to AWS and move their on-premises files to S3 as objects for analytics and machine learning. In 2021, we launched Amazon FSx File Gateway, making it easier for customers to access and use the fully managed Windows-based file storage service, Amazon FSx for Windows File Server, from applications in their data centers. Amazon FSx File Gateway provides low-latency on-premises access to fully managed file shares in the cloud. Customers that want to take advantage of fully managed cloud file storage, but require low latency for their users and applications, can easily extend Amazon FSx for Windows File Server into their existing on-premises environments using FSx File Gateway. The picture below shows the end-to-end FSx File Gateway architecture. To learn more, visit FSx File Gateway launch blog.

As customers used the Amazon FSx File Gateway for user file share use cases, they asked for additional file management features. To enable customers that have needs for larger file share capacities as well as multiple file systems, FSx File Gateway added support for up to 5 file systems and 320-TB total capacity (before deduplication) per gateway. For customers that need to limit storage capacity usage of their on-premises end users, we added support for soft quotas.

Many customers using S3 File Gateway also asked for additional features to meet their security and compliance needs. To address customers’ needs to connect their on-premises S3 File Gateway over a private network connection to S3, we added support for AWS PrivateLink for S3, enabling customers to connect their gateway to S3 without using an HTTP proxy. Along with S3 PrivateLink support, we also added support for S3 Access Points, enabling customers to easily add hundreds of access points without needing to worry about managing access through a single bucket policy that spans hundreds of use cases. Using S3 Access Points in their file shares, customers can create file shares to shared datasets with policies tailored to the specific application.

To address IT administrators’ and compliance managers’ security and compliance needs for audit logs on user access to files and folders, we added support for NFS file share auditing on S3 File Gateway. This enhancement added to the previously released support for SMB file share auditing, making it easier for administrators to log their end users’ key operations for files and folders including create, delete, read, write, rename, and change of permissions.

To provide IT administrators’ flexibility in managing files accidentally left open or locked by a file user on S3 File Gateway, we enabled IT administrators to assign force-closing permissions to users and groups from the connected Active Directory. We also added SMB opportunistic locks or ‘oplocks’ capability on S3 File Gateway, increasing read/write performance for SMB file share users.

Enhanced customer experience and usability

Along with adding functional capabilities, we also enhanced the Storage Gateway console experience for customers making it easier for them to get started with using the cloud for their hybrid cloud storage needs.

We simplified gateway creation and management helping customers speed up their gateway deployment and providing them the same look and feel as they experience with other AWS services. Additionally, we made it easier and faster for customers to manage their tapes. Customers managing hundreds of thousands of tapes can now easily search, view, and manage their tapes stored using common filters such as tape barcode and status.

The picture below from the AWS Storage Gateway management console shows an example of one of the user experience enhancements of using an activation key to activate a gateway.

Expanded compliance support and availability to new AWS Regions

To enable US federal agencies and commercial customers working with the US Federal government that operate under FedRAMP compliance programs, we launched FedRAMP Moderate compliance for Storage Gateway. With this launch, Storage Gateway supports both FedRAMP High and FedRAMP Moderate compliances, enabling customers to store and manage their controlled unclassified information through the gateway.

We expanded Storage Gateway availability in Japan by launching in the AWS Asia Pacific (Osaka) Region.

Summary

In this blog, I covered the new capabilities Storage Gateway launched in 2021 to address our customers’ burgeoning hybrid cloud storage needs. We look forward to hearing from customers on how these new features are benefiting them and building even more capabilities in 2022 to address emerging hybrid and edge cloud storage needs.

To learn more about Storage Gateway, visit Storage Gateway product page and see our Hybrid cloud: Bring AWS to your data centers, remote offices, and beyond and Accelerate physical tape data migration to AWS sessions from re:Invent 2021.

To get started with using AWS Storage Gateway today, visit the Storage Gateway console. Thank you for reading and let me know if you have any comments on the blog or requests for new features.