AWS re:Invent recap: Choosing storage for on-premises file-based workloads

Back in 2016, we first launched File Gateway, a gateway type of the hybrid cloud storage service AWS Storage Gateway. At the time, I had at least one conversation daily (or so it seemed) about File Gateway or Amazon EFS being the right solution for NFS workloads. In 2016, File Gateway only supported NFS, and on the surface seemed to be competitive with Amazon EFS, but each service solved (and continues to solve) different problems. Here we are in 2020, and the conversation is similar with Amazon FSx for Windows File Server (Amazon FSx) and File Gateway for Server Message Block (SMB) workloads. Despite what seems like potential overlap, these services solve different problems and apply to different use cases.

A few days ago, I presented a session at AWS re:Invent 2020-2021: “Choosing the right storage for your on-premises file-based workloads.” In this session, I spoke about AWS Storage for your on-premises file use cases. Workloads relying on the SMB protocol, such as home directories, cloud file shares, content repositories, and business applications can benefit from the scalability, durability, and reliability of AWS Storage solutions. By watching the session, you can understand which service to use in various scenarios, learn from real-world examples, and discover fast and easy ways to accelerate your file migration journey. My 30-minute session is now available on-demand.

In this blog, I provide details and background around File Gateway and Amazon FSx for Windows File Server, recapping my session at re:Invent 2020-2021.

AWS Storage Gateway’s File Gateway

File Gateway enables you to store and access objects in Amazon S3 from file-based applications. There are a lot of applications that can benefit from S3 on the backend, but don’t have native support for the S3 API. This is where File Gateway really shines, enabling access to S3 over file sharing protocols when the application doesn’t have native support for the S3 API.

Several years ago, I worked with a customer that had medical data, and wanted to use Amazon S3 for the scalability, security, and ability to replicate to another AWS Region. The problem for this customer was that the application did not support the S3 API, and only supported NFS for storing and retrieving data. File Gateway was the perfect bridge for this customer as they moved into AWS and modified their application to support the S3 API natively. They were able to move their data into S3, and then use File Gateway to provide access to the data for their application, which was still running in their data center. For this customer and many others, File Gateway was the entry point to AWS, and eventually this customer moved the application into Amazon EC2 and added native S3 support. With File Gateway, the data stays in its native format. This enables you to use all of the applications and tools in the S3 universe to interrogate, manage, and find new value in the data.

Amazon FSx for Windows File Server

Amazon FSx provides native Windows-based SMB file shares for use on premises or inside of AWS. The service provides a fully managed Windows Server, including features like automated backup, patches, security, encryption, monitoring and logging, and redundancy – all for pennies per GB. Amazon FSx also gives you access to SMB, deduplication, compression, encryption, shadow copies, and user quotas, all within AWS using Windows Server at the core. Getting access to all of these features makes it easy to replace a file server environment for SMB clients with a fully managed solution. If you have ever had to patch operating systems, and hope that the server reboots successfully, you know why having a managed environment is so important (and time-saving).

Scalable performance is another highlight of Amazon FSx. File systems can use SDD or HDD storage, and capacity and performance are scalable after you have created a file system. This enables you to provision what you need today, and grow tomorrow without needing to start over or migrate data. You also have PowerShell and the AWS Management Console, which combine for a powerful management tool. In addition, with support for on-premises Active Directory or Active Directory in AWS, you have the flexibility to deploy Amazon FSx at any point in your journey to the cloud.

Conclusion

In this blog post, and in my re:Invent session, I talked about ways to help you choose the right storage for your on-premises file workloads.

When you need to access S3 using a file system protocol, use File Gateway. You get a local cache in the gateway that provides high throughput and low latency over SMB. Your users can mount the share and access data in the S3 bucket, and applications that support the native S3 API can continue to work that way. Older applications that don’t support the S3 API can start using S3 today with SMB or NFS.

For your native Windows workloads and users, or your SMB clients, Amazon FSx for Windows File Server provides all of the benefits of a native Windows SMB environment that is fully managed and secured, and scaled like any other AWS service. You get detailed reporting, replication, backup, failover, and support for native Windows tools like DFS and Active Directory.

The conversation isn’t about using File Gateway or Amazon FSx for Windows, the conversation is about what you are trying to accomplish. Getting started is easy, just log in to the AWS Management Console, select the Amazon FSx console or the AWS Storage Gateway console under the Storage category, and start building.

Feel free to leave any comments or questions about my re:Invent session or this blog post in the comments section. Thank you for reading!