Introducing AWS Storage Gateway
AWS Storage Gateway is a hybrid cloud storage service that gives you on-premises access to virtually unlimited cloud storage. You can use Storage Gateway to simplify storage management and reduce costs for key hybrid cloud storage use cases. These include moving backups to the cloud, using on-premises file shares backed by cloud storage, and providing low-latency access to data in AWS for on-premises applications.
To support these use cases, the service provides four different types of gateways – Tape Gateway, Amazon S3 File Gateway, Amazon FSx File Gateway, and Volume Gateway – that seamlessly connect on-premises applications to cloud storage, caching data locally for low-latency access.
Storage Gateway is fast and easy to deploy, enabling you to integrate it with your existing environments and access AWS Storage in a frictionless manner. The service also provides a consistent management experience using the AWS Management Console, both for on-premises gateways, and for monitoring, management, and security with AWS services such as Amazon CloudWatch, AWS CloudTrail, AWS Identity and Access Management (IAM), and AWS Key Management Service (KMS). Storage Gateway helps you reduce cost, maintenance, and scaling challenges associated with managing on-premises storage environments.
Standard Storage Protocols: Storage Gateway seamlessly connects to your local production or backup applications with NFS, SMB, iSCSI, or iSCSI-VTL, so you can adopt AWS Cloud storage without needing to modify your applications. Its protocol conversion and device emulation enables you to access block data on volumes managed by Storage Gateway on top of Amazon Simple Storage Service (S3), store files as native Amazon S3 objects or in fully managed cloud file shares with Amazon FSx for Windows File Server, and keep virtual tape backups online in a virtual tape library backed by S3 or move the backups to a tape archive tier on Amazon S3 Glacier and Amazon S3 Glacier Deep Archive.
Fully Managed Cache: The local gateway appliance maintains a cache of recently written or read data so your applications can have low-latency access to data that is stored durably in AWS. The gateways use a read-through and write-back cache, committing data locally, acknowledging the write operations, and then asynchronously copying data to AWS, reducing application latency.
Optimized and Secure Data Transfer: Storage Gateway provides secure upload of changed data and secure downloads of requested data, encrypting data in transit between any type of gateway appliance and AWS using SSL. Storage Gateway delivers end-to-end protection of customer data from the Storage Gateway in the enterprise network to the data residing in AWS. The service supports security features and access controls, and supplies compliances and certifications that address enterprise customers’ real and perceived security concerns when using AWS storage via the Storage Gateway Optimizations such as multi-part management, automatic buffering, delta transfers used across all gateway types, and data compression applied for all block and virtual tape data. Storage Gateway offers Federal Information Processing Standard 140-2 (FIPS) compliant endpoints in AWS GovCloud (US-East) and AWS GovCloud (US-West).
Storage Gateway enables you to easily consume AWS services. As a native AWS service, Storage Gateway integrates with other AWS services for storage, backup, and management while still integrating with on-premises environments. The service stores files as native Amazon S3 objects or fully managed file shares in Amazon FSx for Windows File Server, archives virtual tapes in Amazon S3 Glacier and Amazon S3 Glacier Deep Archive, and stores EBS snapshots generated by the Volume Gateway with Amazon EBS. Storage Gateway also integrates with AWS Backup to manage backup and recovery of Volume Gateway volumes, simplifying your backup management, and helping you meet your business and regulatory backup compliance requirements. Storage Gateway publishes health and performance logs and metrics to Amazon CloudWatch and provides monitoring of metrics and alarms in the Storage Gateway console. Storage Gateway integrates with AWS IAM to help manage and secure access to Storage Gateway resources. Your data is encrypted by default at rest using S3-SSE or you can choose to use your own encryption keys through Storage Gateway's integration with AWS KMS.
High Availability on VMware: Storage Gateway provides high availability on VMware through a set of health-checks integrated with VMware vSphere High Availability (VMware HA). With this integration, Storage Gateway deployed in a VMware environment on-premises, or in VMware Cloud on AWS, will automatically recover from most service interruptions in under 60 seconds. This protects storage workloads against hardware, hypervisor, or network failures; storage errors; and software errors, such as connection timeouts and file share or volume unavailability.
Amazon S3 File Gateway
Amazon S3 File Gateway presents a file interface that enables you to store files as objects in Amazon S3 using the industry-standard NFS and SMB file protocols, and access those files via NFS and SMB from your data center or Amazon EC2, or access those files as objects directly in Amazon S3. POSIX-style metadata, including ownership, permissions, and timestamps are durably stored in Amazon S3 in the user-metadata of the object associated with the file. Once objects are transferred to S3, they can be managed as native S3 objects and bucket policies such as lifecycle management and Cross-Region Replication (CRR), and can be applied directly to objects stored in your bucket. Amazon S3 File Gateway also publishes audit logs for SMB file share user operations to Amazon CloudWatch.
Customers can use Amazon S3 File Gateway to back up on-premises file data as objects in Amazon S3 (including Microsoft SQL Server and Oracle databases and logs), and for hybrid cloud workflows using data generated by on-premises applications for processing by AWS services such as machine learning or big data analytics.
Amazon FSx File Gateway
Amazon FSx File Gateway provides fast, low-latency on-premises access to fully managed, highly reliable, and scalable file shares in the cloud using the industry-standard SMB protocol. Customers can store and access file data in Amazon FSx with Windows-native compatibility including full NTFS support, shadow copies, and Access Control Lists (ACLs). Use Amazon FSx File Gateway for your on-premises file-based business applications and workloads such as user or group file shares, web content management, and media workflows.
With Amazon FSx File Gateway, you can easily migrate and consolidate your on-premises file-based application data stored on Network-Attached-Storage (NAS) arrays or file server VMs into FSx for Windows File Server for scalable shared file access that seamlessly integrates with your existing environment. With the HDD file storage option, Amazon FSx for Windows File Server offers the lowest-cost file storage in the cloud for Windows applications and workloads, or SSD storage for performance-intensive workloads.
Customers that use Amazon FSx File Gateway can also benefit from other integrated AWS services for simplified storage management and data protection. You can automatically send logs of SMB user operations to Amazon CloudWatch to perform auditing and analysis, and use AWS Backup for centralized backup and retention.
Tape Gateway presents an iSCSI-based virtual tape library (VTL) of virtual tape drives and a virtual media changer to your on-premises backup application. It is compatible with most leading backup applications, so you can continue using your tape-based backup workflows. Tape Gateway stores your virtual tapes in Amazon Simple Storage Service (Amazon S3) and creates new ones automatically, simplifying management and your transition to AWS.
Its VTL interface helps you reduce physical tape infrastructure capital expenses, multi-year maintenance contract commitments, and ongoing media costs. You pay only for the capacity you use and scale as your needs grow. For any petabyte-scale tape data migration needs you can use a Snowball Edge Storage Optimized device with Tape Gateway to move your physical tape data to either S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive, further reducing your long-term storage costs. You can access your data stored as virtual tapes in AWS through a Tape Gateway running in AWS or in your data center over the network.
With Tape Gateway, you’ll no longer need to store media at offsite facilities and migrate tape media from one generation to the next manually.
Volume Gateway presents your applications' block storage volumes using the iSCSI protocol. Data written to these volumes can be asynchronously backed up as point-in-time snapshots of your volumes, and stored in the cloud as Amazon EBS snapshots. You can back up your on-premises Volume Gateway volumes using the service’s native snapshot scheduler or by using the AWS Backup service. In both cases, volume backups are stored as Amazon EBS snapshots in AWS. These snapshots are incremental backups that capture only changed blocks. All snapshot storage is also compressed to minimize your storage charges.
Customers often choose Volume Gateway to back up local applications, and use it for disaster recovery based on EBS Snapshots, or Cached Volume Clones. Volume Gateway integration with AWS Backup enables customers to use the AWS Backup service to protect on-premises applications that use Storage Gateway volumes. AWS Backup supports backup and restore of both cached and stored volumes. Using AWS Backup with Volume Gateway together helps you centralize backup management, reduce your operational burden, and meet compliance requirements.
Storage Gateway Deployment Options
As a hybrid cloud service, AWS Storage Gateway is fully managed and consists of both cloud and on-premises components, which can be deployed in several methods based on your on-premises infrastructure needs: as a virtual machine, which can run on VMware ESXi, Microsoft Hyper-V, or Linux KVM on premises; as a hardware appliance on-premises; as a VM in VMware Cloud on AWS; or as an AMI in Amazon EC2.
Storage Gateway provides public, Amazon VPC, and FIPS service endpoints, providing you options to deploy and connect your gateway to Storage Gateway in a framework that best suits your networking and security needs. You can connect a gateway to the service either using public internet or through AWS Direct Connect.