AWS Storage Gateway Documentation
Introducing AWS Storage Gateway
AWS Storage Gateway is a hybrid cloud storage service that is designed to give you on-premises access to virtually unlimited cloud storage. Customers use Storage Gateway to simplify storage management and reduce costs for key hybrid cloud storage use cases. These include moving backups to the cloud, using on-premises file shares backed by cloud storage, and providing low latency access to data in AWS for on-premises applications.
To support these use cases, the service provides four different types of gateways—Tape Gateway, Amazon S3 File Gateway, Amazon FSx File Gateway, and Volume Gateway—that are designed to connect on-premises applications to cloud storage, caching data locally for low-latency access.
Key Features
Storage Gateway helps enable you to integrate with your existing environments and access AWS Storage. The service also provides a consistent management experience using the AWS Console, both for on-premises gateways, and for monitoring, management, and security with AWS services such as Amazon CloudWatch, AWS CloudTrail, AWS Identity and Access Management (IAM), and AWS Key Management Service (KMS). Storage Gateway helps you reduce cost, maintenance, and scaling challenges associated with managing on-premises storage environments.
Standard Storage Protocols: Storage Gateway is designed to connect to your local production or backup applications with NFS, SMB, iSCSI, or iSCSI-VTL, so you can adopt AWS Cloud storage without needing to modify your applications. Its protocol conversion and device emulation are designed so you can access block data on volumes managed by Storage Gateway on top of Amazon S3, store files as native Amazon S3 objects or in fully managed cloud file shares with Amazon FSx for Windows File Server, and keep virtual tape backups online in a virtual tape library backed by Amazon S3. You can also move backups to a tape archive tier on Amazon S3 Glacier or Amazon S3 Glacier Deep Archive.
Caching: The local gateway appliance is designed to maintain a cache of recently written or read data so your applications can have low-latency access to data that is stored durably in AWS. The gateways are designed to use a read-through and write-back cache, committing data locally, acknowledging the write operations, and then asynchronously copying data to AWS, reducing application latency.
Optimized and Secure Data Transfer: Storage Gateway is designed to help you secure upload of changed data and secure downloads of requested data, encrypting data in transit between any type of gateway appliance and AWS using SSL. Storage Gateway is also designed to help you with end-to-end protection of customer data from the Storage Gateway in the enterprise network to the data residing in AWS. The service supports security features, access controls, and supplies compliances and certifications that address enterprise customers’ real and perceived security concerns when using AWS Cloud storage via the Storage Gateway Optimizations such as multi-part management, automatic buffering, delta transfers used across all gateway types, and data compression applied for all block and virtual tape data.
Storage Gateway enables customers to easily consume AWS services. As a native AWS service, Storage Gateway integrates with other AWS services for storage, backup, and management while still integrating with on-premises environments. The service stores files as native Amazon S3 objects or fully managed file shares in Amazon FSx for Windows File Server, archives virtual tapes in Amazon S3 Glacier or Amazon S3 Glacier Deep Archive, and stores EBS snapshots generated by the Volume Gateway with Amazon EBS. Storage Gateway also integrates with AWS Backup to manage backup and recovery of Volume Gateway volumes, simplifying your backup management, and helping you meet your business and regulatory backup compliance requirements. Storage Gateway publishes health and performance logs and metrics to Amazon CloudWatch and provides monitoring of metrics and alarms in the Storage Gateway console. Storage Gateway integrates with AWS IAM to help manage and secure access to Storage Gateway resources. Storage Gateway supports encryption at rest by default using S3-SSE or you can choose to use your own encryption keys through Storage Gateway's integration with AWS KMS.
High Availability on VMware: Storage Gateway provides high availability on VMware through a set of health-checks integrated with VMware vSphere High Availability (VMware HA). With this integration, Storage Gateway deployed in a VMware environment on-premises, or in VMware Cloud on AWS, is designed to recover from most service interruptions in under 60 seconds. This helps protect storage workloads against hardware, hypervisor, or network failures, and storage or software errors, such as connection timeouts and file share or volume unavailability.
Gateway Types
Amazon S3 File Gateway
Amazon S3 File Gateway presents a file interface that enables you to store files as objects in Amazon S3 using the industry-standard NFS and SMB file protocols, and access those files via NFS and SMB from your data center or Amazon EC2, or access those files as objects directly in Amazon S3. POSIX-style metadata, including ownership, permissions, and timestamps, are durably stored in Amazon S3 in the user-metadata of the object associated with the file. Once objects are transferred to S3, they can be managed as native S3 objects and bucket policies such as lifecycle management and Cross-Region Replication (CRR), and can be applied directly to objects stored in your bucket. Amazon S3 File Gateway also publishes audit logs for SMB file share user operations to Amazon CloudWatch.
Customers can use Amazon S3 File Gateway to back up on-premises file data as objects in Amazon S3 (including Microsoft SQL Server and Oracle databases and logs), and for hybrid cloud workflows using data generated by on-premises applications for processing by AWS services such as machine learning or big data analytics.
Amazon FSx File Gateway
Amazon FSx File Gateway provides fast, low-latency on-premises access to fully managed, highly reliable, and scalable file shares in the cloud using the industry-standard SMB protocol. Customers can store and access file data in Amazon FSx with Windows-native compatibility including full NTFS support, shadow copies, and Access Control Lists (ACLs). Use Amazon FSx File Gateway for your on-premises file-based business applications and workloads such as user or group file shares, web content management, and media workflows.
With Amazon FSx File Gateway, customers can migrate and consolidate their on-premises file-based application data stored on Network-Attached-Storage (NAS) arrays or file server VMs into FSx for Windows File Server for scalable shared file access that integrates with your existing environment. With the HDD file storage option, Amazon FSx for Windows File Server offers low-cost file storage in the cloud for Windows applications and workloads, or SSD storage for performance-intensive workloads.
Customers that use Amazon FSx File Gateway can also benefit from other integrated AWS services for simplified storage management and data protection. You can send logs of SMB user operations to Amazon CloudWatch to perform auditing and analysis, and use AWS Backup for centralized backup and retention.
Tape Gateway
Tape Gateway presents a virtual tape library (VTL) consisting of virtual tape drives and a virtual media changer to your backup application using storage industry standard iSCSI protocol. You can continue to use your existing backup applications and workflows while writing to virtual tapes. Each virtual tape is stored in Amazon S3. When you no longer require immediate or frequent access to data contained on a virtual tape, you can have your backup application move it from the Storage Gateway Virtual Tape Library into an archive tier that sits on top of Amazon S3 Glacier or Amazon S3 Glacier Deep Archive cloud storage, further reducing storage costs. Tape Gateway stores your virtual tapes in service-managed S3 buckets, and can create new virtual tapes, simplifying management and making your transition to the cloud for storage easy.
Volume Gateway
Volume Gateway presents your application’s block storage volumes using the iSCSI protocol. Data written to these volumes can be asynchronously backed up as point-in-time snapshots of your volumes, and stored in the cloud as Amazon EBS snapshots. You can back up your on-premises Volume Gateway volumes using the service’s native snapshot scheduler or by using the AWS Backup service. In both cases, volume backups are stored as Amazon EBS snapshots in AWS. These snapshots are incremental backups that capture only changed blocks. All snapshot storage is also compressed to minimize your storage charges.
Customers often choose Volume Gateway to backup local applications, and use it for disaster recovery based on EBS Snapshots, or Cached Volume Clones. Volume Gateway integration with AWS Backup enables customers to use the AWS Backup service to protect on-premises applications that use Storage Gateway volumes. AWS Backup supports backup and restore of both cached and stored volumes. Using AWS Backup with Volume Gateway together helps you centralize backup management, reduce your operational burden, and meet compliance requirements.
Storage Gateway Deployment Options
As a hybrid cloud service, AWS Storage Gateway consists of in-cloud as well as on-premises components, which can be deployed in several methods based on your on-premises infrastructure needs: as a virtual machine, as a hardware appliance on-premises, as a VM in VMware Cloud on AWS, or as an AMI in Amazon EC2.
Storage Gateway provides public, Amazon VPC, and FIPS service endpoints, providing you options to deploy and connect your gateway to Storage Gateway in a framework that best suits your networking and security needs. You can connect a gateway to the service either using public internet or through AWS Direct Connect.
For additional information about service controls, security features and functionalities, including, as applicable, information about storing, retrieving, modifying, restricting, and deleting data, please see https://docs.aws.amazon.com/index.html. This additional information does not form part of the Documentation for purposes of the AWS Customer Agreement available at http://aws.amazon.com/agreement, or other agreement between you and AWS governing your use of AWS’s services.