More files, more problems?
Nearly all enterprises, regardless of industry, have to store files, whether they are backups, media content or specialized vertical application datasets. Managing and scaling on-premises infrastructure to provide online storage and distribution of such backup or content files is often burdensome and costly, requiring expensive hardware refreshes, expansion and software licensing. Such large file data repositories can be siloed in specialized file servers, NAS units or backup systems, limiting access for big data analytics or media processing applications.
AWS Storage Gateway's file interface, or file gateway, offers you a seamless way to connect to the cloud in order to store application data files and backup images as durable objects on Amazon S3 cloud storage. File gateway offers SMB or NFS-based access to data in Amazon S3 with local caching. It can be used for on-premises applications, and for Amazon EC2-resident applications that need file storage in S3 for object based workloads.
Why use AWS Storage Gateway and Amazon S3 for file storage
Easy hybrid cloud connection
Traditional SMB, NFS, or S3 API interfaces
Global access & distribution
Scalability & flexibility of AWS
Once the file gateway has moved data into Amazon S3, you can manipulate, analyze and manage it using native AWS services via API. Additionally, from your Amazon S3 bucket, you can distribute that data to other regions around the world with Cross-Region Replication, apply storage management tools, use Lifecycle policies to migrate it to archive-tier Amazon Glacier cloud storage, and even deploy additional file gateways to access it from your other sites.
File gateway use cases
Online content repository
The file gateway allows you to cost-effectively and durably store large files and media assets on AWS. Local applications also benefit from a low-latency local cache of frequently used content. The result is tiered, hybrid cloud content storage, which can be accessed easily by on-premises applications via NFS or SMB, from wherever you deploy gateway appliances - including in Amazon EC2. Storage Gateway automatically preserves the file metadata as object metadata, and also preserves the directory structure by including it in the object name. Content stored in Amazon S3 can be manipulated by in-cloud services via API, for example to do automatic image resizing with an AWS Lambda function, or to index the files with Amazon Elasticsearch Service.
Backup to cloud
Many organizations start their cloud journey by moving secondary and tertiary data, such as backups, to the cloud. The file gateway’s SMB and NFS interfaces provide an ideal way for IT groups to simply transition their backup jobs from existing on-premises backup systems to the cloud. Backup applications, native database tools or scripts that can write to SMB or NFS can write to the file gateway, which will store the backups as Amazon S3 objects of up to 5TiB in size. With an adequately sized local cache, recent backups can be used for fast on-site recoveries, while long-term retention needs are addressed by tiering backups to low-cost S3 Standard, S3 Infrequent Access and Amazon Glacier cloud storage tiers.
Big data, machine learning & processing
The file gateway makes it easy for Business Intelligence, Analytics or other teams that use Machine Learning to easily move file-based data into Amazon S3. They can then use that data for analytics, either with in-place queries via services such as Amazon Athena, Amazon Redshift Spectrum, or load it into other cloud tools such Amazon EMR for Hadoop-based processing. Post-analysis, result sets can be stored back in the same bucket, and the storage gateway service can make those new results files (objects) visible to on-premises applications wherever you have deployed a file gateway.
Additionally, you can apply simple compute functions with AWS Lambda to process data files stored in S3 with the file gateway, or even apply Machine Learning services to the data, for instance using Amazon Rekognition to perform image recognition or flag objectionable content.
Vertical industry applications
Industries including Oil & Gas, Media & Entertainment, Design & Architecture, and Manufacturing have domain-specific applications that generate large specialized files. These files often need to be distributed, or at least accessible, from multiple sites. Over time, most of these files become infrequently accessed, and can be stored on lower cost, but durable online cloud storage, if not fully archived. File gateway allows such on-premises applications, across multiple locations, to use Amazon S3 and Amazon Glacier to store the files. It also enables migration of such file-based applications to Amazon EC2, by providing the central, globally accessible repository, based on Amazon S3 object storage.
Results that file gateway and AWS deliver
IT outcomes
- Reduce datacenter infrastructure footprint; Minimize storage and backup stacks
- Focus on strategic initiatives, applications, optimal architectures, and flexible efficient operations
- Reduce the operational burden of maintaining and refreshing hardware
- Global scalability with easy redundancy - without infrastructure management
- Data durability: Amazon S3 and Amazon Glacier cloud storage is designed for 99.999999999% of durability
Business results
- Flexibility to evolve business as needed
- Shift to OpEx and consumption purchasing model that can be aligned to line of business growth
- Eliminate large and unexpected CapEx outlays
- Lower total costs of storing and processing data
- Confidence that critical data is safe and secure, and that your organization is in compliance with necessary regulations
Featured customers
“Our immersive digital strategy is enabling us to exploit the immense potential of mRNA science to deliver transformative medicines for many diseases, helping position us as one of today’s most notable high-growth biotechs. Seamlessly integrated and orchestrated cloud-based IT systems are critical to manage and industrialize the complex planning and execution of our mRNA pipeline scale-up at every stage of drug development. AWS Storage Gateway has the promise to transform the way we move data into the cloud. The file interface lets us easily integrate data files from analytical instruments, and the transparent S3 storage lets us easily connect our cloud-based applications and leverage the powerful storage capabilities of S3. With the AWS File Gateway, we can now unleash the full power of AWS on our instrument data.”
-- Dave Johnson, PhD, Director, Informatics - Moderna Therapetuics
Learn more: how it works
AWS Storage Gateway resources
Learn more about AWS Storage Gateway pricing
