Introducing Storage Gateway
AWS Storage Gateway is a hybrid cloud storage service that connects your existing on-premises environments with the AWS Cloud. Its features make it easy for you to run hybrid cloud workloads at any stage of your cloud adoption, whether it's getting started with cloud backups, running cloud processing workflows for data generated by on-premises machines, or performing a one-time migration of block volume data or databases.
Standard Storage Protocols. Storage Gateway seamlessly connects to your local production or backup applications with NFS, SMB, iSCSI, or iSCSI-VTL, so you can adopt AWS Cloud storage without needing to modify your applications. Its protocol conversion and device emulation enables you to access block data on volumes managed by Storage Gateway on top of Amazon S3, store files as native Amazon S3 objects, and keep virtual tape backups online in a Virtual Tape Library backed by S3 or move the backups to a tape archive tier on Amazon S3 Glacier.
Fully Managed Cache. The local gateway appliance maintains a cache of recently written or read data so your applications can have low-latency access to data that is stored durably in AWS. The gateways use a read-through and write-back cache.
Optimized and Secured Data Transfer. Storage Gateway provides secure upload of changed data and secure downloads of requested data, encrypting data in transit between any type of gateway appliance and AWS using SSL. Optimizations such as multi-part management, automatic buffering, and delta transfers are used across all gateway types, and data compression is applied for all block and virtual tape data.
AWS Integrated. As a native AWS service, Storage Gateway integrates with other AWS services for storage, backup, and management. The service stores files as native Amazon S3 objects, archives virtual tapes in Amazon S3 Glacier, and stores EBS Snapshots generated by the Volume Gateway with Amazon EBS. Storage Gateway also integrates with AWS Backup to manage backup and recovery of Volume Gateway volumes, simplifying your backup management, and helping you meet your business and regulatory backup compliance requirements.
High Availability on VMware. Storage Gateway provides high availability on VMware through a set of health-checks integrated with VMware vSphere High Availability (VMware HA). With this integration, a Storage Gateway deployed in a VMware environment on-premises, or in VMware Cloud on AWS, will automatically recover from most service interruptions in under 60 seconds. This protects storage workloads against hardware, hypervisor, or network failures, storage errors, or software errors, such as connection timeouts and file share or volume unavailability.
Additionally, Storage Gateway provides a consistent management experience using the AWS Console, both for on-premises gateways, and for monitoring, management, and security with AWS services such as Amazon CloudWatch, AWS CloudTrail, AWS Identity and Access Management (IAM), and AWS Key Management Service (KMS).
The File Gateway presents a file interface that enables you to store files as objects in Amazon S3 using the industry-standard NFS and SMB file protocols, and access those files via NFS and SMB from your datacenter or Amazon EC2, or access those files as objects with the S3 API. POSIX-style metadata, including ownership, permissions, and timestamps are durably stored in Amazon S3 in the user-metadata of the object associated with the file. Once objects are transferred to S3, they can be managed as native S3 objects, and bucket policies such as versioning, lifecycle management, and cross-region replication and apply directly to objects stored in your bucket.
Customers use the File Gateway to store file data into S3 for use by object-based workloads including data analytics or machine learning, as a cost-effective storage target for backups, and as a repository or tier in the cloud for application file storage.
The Tape Gateway presents itself to your existing backup application as an industry-standard iSCSI-based virtual tape library (VTL), consisting of a virtual media changer and virtual tape drives. You can continue to use your existing backup applications and workflows while writing to a nearly limitless collection of virtual tapes. Each virtual tape is stored in Amazon S3. When you no longer require immediate or frequent access to data contained on a virtual tape, you can have your backup application move it from the Storage Gateway Virtual Tape Library into an archive tier that sits on top of Amazon S3 Glacier cloud storage, further reducing storage costs.
Storage Gateway is currently compatible with most leading backup applications. The Tape Gateway’s VTL interface eliminates large upfront tape automation capital expenses, multi-year maintenance contract commitments, and ongoing media costs. You pay only for the capacity you use and scale as your needs grow. The need to transport storage media to offsite facilities and handle tape media manually goes away, and your archives benefit from the design and durability of the AWS Cloud platform.
The Volume Gateway presents your applications block storage volumes using the iSCSI protocol. Data written to these volumes can be asynchronously backed up as point-in-time snapshots of your volumes, and stored in the cloud as Amazon EBS snapshots. You can back up your on-premises Volume Gateway volumes using the service’s native snapshot scheduler or the AWS Backup service. In both cases, volume backups are stored as Amazon EBS snapshots in AWS. These snapshots are incremental backups that capture only changed blocks. All snapshot storage is also compressed to minimize your storage charges.
When connecting to the Volume Gateway with the iSCSI block interface, you can run the gateway in two modes: cached and stored. In cached mode, you store your primary data in Amazon S3 and retain your frequently accessed data locally in cache. With this mode, you can achieve substantial cost savings on primary storage, minimizing the need to scale your storage on-premises, while retaining low-latency access to your frequently accessed data.
In stored mode, you store your entire data set locally, while making an asynchronous copy of your volume in Amazon S3 and point-in-time EBS snapshots. This mode provides durable and inexpensive offsite backups that you can recover locally, to another site or in Amazon EC2.
Customers often choose the volume gateway to backup local applications, and use it for disaster recovery based on EBS Snapshots, or Cached Volume Clones. The Volume Gateway integration with AWS Backup enables customers to use the AWS Backup service to protect on-premises applications that use Storage Gateway volumes. AWS Backup supports backup and restore of both cached and stored volumes. Using AWS Backup with Volume Gateway helps you centralize backup management, reduce your operational burden, and meet compliance requirements. AWS Backup enables you to:
- Set customizable scheduled backup policies that meet your backup requirements
- Set backup retention and expiration rules, so you no longer need to develop custom scripts or manually manage the point-in-time backups of your volumes
- Manage and monitor backups across multiple gateways and other AWS resources from a central view
Storage Gateway Deployment Options
The AWS Storage Gateway service consists of its in-cloud components, including the management console, storage infrastructure and back-end control and integration services and APIs, and the gateway appliance that you deploy and connect to your applications.
You have four options for deployment: either a virtual machine containing the Storage Gateway software, which can run on VMware ESXi, Microsoft Hyper-V on premises, as a hardware appliance on-premises, as a VM in VMware Cloud on AWS, or as an AMI in Amazon EC2.
Storage Gateway as a hardware appliance
Storage Gateway is available pre-installed on a hardware appliance, a Dell EMC PowerEdge R640XL server with a validated configuration. The hardware appliance provides a simple procurement, deployment, and management experience for customers who have limited virtualized infrastructure, burdensome centralized resource provisioning processes, or limited IT staffing.