Q. What is the AWS Storage Gateway and what can I do with it?
The AWS Storage Gateway is a service connecting an on-premises software appliance with cloud-based storage to provide seamless and secure integration between an organization’s on-premises IT environment and AWS’s storage infrastructure. The service enables you to securely store data to the AWS cloud for scalable and cost-effective storage. The AWS Storage Gateway supports industry-standard storage protocols that work with your existing applications. It provides low-latency performance by maintaining frequently accessed data on-premises while securely storing all of your data encrypted in the Amazon Simple Storage Service (Amazon S3).
The AWS Storage Gateway supports two configurations:
Gateway-Cached Volumes: You can store your primary data in Amazon S3, and retain your frequently accessed data locally. Gateway-Cached volumes provide substantial cost savings on primary storage, minimize the need to scale your storage on-premises, and retain low-latency access to your frequently accessed data.
Gateway-Stored Volumes: In the event you need low-latency access to your entire data set, you can configure your gateway to store your primary data locally, and asynchronously back up point-in-time snapshots of this data to Amazon S3. Gateway-Stored volumes provide durable and inexpensive off-site backups that you can recover locally or from Amazon EC2 if, for example, you need replacement capacity for disaster recovery.
Q. How does the AWS Storage Gateway work?
The AWS Storage Gateway’s software appliance is available for download as a virtual machine (VM) image that you install on a host in your datacenter. Once you’ve installed your gateway and associated it with your AWS Account through our activation process, you can use the AWS Management Console to create either Gateway-Cached or Gateway-Stored storage volumes that can be mounted as iSCSI devices by your on-premises applications.
Gateway-Cached volumes allow you to utilize Amazon S3 for your primary data, while retaining some portion of it locally in a cache for frequently accessed data. These volumes minimize the need to scale your on-premises storage infrastructure, while still providing your applications with low-latency access to their frequently accessed data. You can create storage volumes up to 32 TBs in size and mount them as iSCSI devices from your on-premises application servers. Data written to these volumes is stored in Amazon S3, with only a cache of recently written and recently read data stored locally on your on-premises storage hardware.
Gateway-Stored volumes store your primary data locally, while asynchronously backing up that data to AWS. These volumes provide your on-premises applications with low-latency access to their entire data sets, while providing durable, off-site backups. You can create storage volumes up to 1 TB in size and mount them as iSCSI devices from your on-premises application servers. Data written to your Gateway-Stored volumes is stored on your on-premises storage hardware, and asynchronously backed up to Amazon S3 in the form of Amazon EBS snapshots.
Q. How can I get started using the AWS Storage Gateway?
To get started, sign up for the AWS Storage Gateway by clicking the “Sign Up Now” button on the AWS Storage Gateway detail page. To sign-up, you must have an Amazon Web Services account; if you do not already have one, you will be prompted to create one when you begin the AWS Storage Gateway sign-up process. After you sign up, you can begin setting up and activating your gateway by visiting the AWS Management Console. To learn more, you can also refer to our Getting Started Video or Getting Started Documentation.
Q. Where does the AWS Storage Gateway store my data in AWS?
Gateway-Stored volumes store your primary data on your on-premises storage hardware, and asynchronously back up this data to Amazon S3 in the form of point-in-time snapshots. Gateway-Cached volumes store your primary data in Amazon S3, while maintaining your frequently accessed data on your local hardware. You can also take point-in-time snapshots of your Gateway-Cached volumes in Amazon S3, enabling you to preserve versions of your data. Amazon S3 stores data in multiple facilities and on multiple devices within each facility. Amazon S3 also performs regular, systematic data integrity checks and is built to be automatically self-healing.
Q. What are the minimum hardware requirements for the AWS Storage Gateway’s VM?
The AWS Storage Gateway VM must be installed on a host in your datacenter running supported versions of VMWare ESXi or Microsoft Hyper-V. The gateway VM must be deployed with these minimum resources:
4 Virtual Processors.
7.5GB of RAM.
75GB of disk space for installation of virtual machine image and system data.
The AWS Storage Gateway currently supports these versions of the Microsoft Windows and Red Hat iSCSI Software Initiators.
Q. How much data can I store per gateway? How many volumes can I create?
Each Gateway-Cached gateway supports up to 150 TBs of storage volumes. Each Gateway-Stored gateway supports up to 12TBs of storage volumes. There is no limit to the amount of Amazon EBS snapshot storage that a single gateway can store in Amazon S3.
Q. What performance will I experience when using the AWS Storage Gateway?
As the AWS Storage Gateway VM sits between your application, Amazon S3, and underlying on-premises storage, the performance you experience will be dependent upon a number of factors, including the speed and configuration of your underlying local disks, the network bandwidth between your iSCSI initiator and gateway VM, the amount of local storage allocated to the gateway VM, and the bandwidth between the gateway VM and Amazon S3. For Gateway-Cached volumes, to provide low-latency read access to your on-premises applications, it’s important that you provide enough local cache storage to store your recently accessed data. Our technical documentation provides guidance on how to optimize your environment setup for best performance, including how to properly size your local storage.
Q. Can I route my AWS Storage Gateway internet traffic through a local proxy server?
Yes, the AWS Storage Gateway supports the configuration of a SOCKS proxy between your gateway and AWS. You can specify an IP address and Port number for the host running your proxy, and the AWS Storage Gateway will route all HTTPS traffic through your proxy server.
Q. Can I use the AWS Storage Gateway with AWS Direct Connect?
The AWS Storage Gateway efficiently uses your internet bandwidth to speed up the upload of your on-premises application data to AWS. The AWS Storage Gateway only uploads data that has changed, minimizing the amount of data sent over the internet. You can also use AWS Direct Connect to further increase throughput and reduce your network costs by establishing a dedicated network connection between your on-premises gateway and AWS.
Q. Can I configure the AWS Storage Gateway via an API?
Yes, you can configure the AWS Storage Gateway via both an API as well as the AWS Management Console.
Q. How will I be billed for my use of the AWS Storage Gateway?
You are billed based on:
Gateway usage (per activated gateway per month): You are billed a monthly fee for each of your gateways. This fee is prorated daily. Billing for a gateway begins upon activation and continues until you delete the gateway from the AWS Management Console or via the API.
Snapshot Storage usage (per GB per month): You are billed for the snapshots your gateway stores in Amazon S3. These snapshots are stored and billed as Amazon EBS snapshots. Snapshots are incremental backups, reducing your storage charges. When taking a new snapshot, only the data that has changed since your last snapshot is stored. All snapshot storage is also compressed, further minimizing your storage charges.
Volume storage usage (per GB per month): You are billed for the Gateway-Cached volume data that you store in Amazon S3. You are only billed for what you use, not what you provision.
Data transfer (per GB) – You are billed for internet data transfer for each GB downloaded from AWS to your gateway. All data transfer in is $0.00 per GB.
Except as otherwise noted, our prices are exclusive of applicable taxes and duties, including VAT and applicable sales tax.
Q. Does the AWS Storage Gateway encrypt my data?
The AWS Storage Gateway encrypts all data in-transit to and from AWS via SSL. All volume and snapshot data is encrypted at rest in Amazon S3 using Advanced Encryption Standard (AES) 256, a secure symmetric-key encryption standard using 256-bit encryption keys. This encryption does not apply to any snapshot data you have stored from your Amazon EBS volumes.
Q. What form of iSCSI authentication does the AWS Storage Gateway support?
The AWS Storage Gateway supports authentication between your gateway and iSCSI initiators via CHAP (Challenge-Handshake Authentication Protocol).
Q. What value do snapshots provide?
Whether you’re using Gateway-Cached or Gateway-Stored volumes, you can take point-in-time, incremental snapshots of your volume and store them in Amazon S3 in the form of Amazon EBS snapshots. For Gateway-Stored volumes, where your volume data is stored on-premises, snapshots provide durable, off-site backups in Amazon S3. You can create a new Gateway-Stored volume from a snapshot in the event you need to recover a backup. You can also use a snapshot of your Gateway-Stored volume as the starting point for a new Amazon EBS volume which you can then attach to an Amazon EC2 instance.
For Gateway-Cached volumes, where your volume data is already stored in Amazon S3, snapshots can be used to preserve versions of your data, allowing you to recover from unintended user actions. You can create a new Gateway-Cached volume from a snapshot in the event you need to recover a prior version.
Snapshots can be initiated on a scheduled or ad-hoc basis. When taking a new snapshot, only the data that has changed since your last snapshot is stored. If you have a volume with 100 GBs of data, but only 5 GBs of data have changed since your last snapshot, only the 5 additional GBs of snapshot data will be stored in Amazon S3. When you delete a snapshot, only the data not needed for any other snapshot is removed.
Q. How do I restore a snapshot to a gateway storage volume?
Using the AWS Management Console, you can create a new gateway volume from a snapshot you’ve stored in Amazon S3. You can then mount this volume as an iSCSI device to your on-premises application server.
Because Gateway-Stored volumes store your primary data locally, when creating a new volume from a snapshot, your gateway downloads the data contained within the snapshot to your local hardware, where it becomes the primary data for your new volume.
Because Gateway-Cached volumes store your primary data in Amazon S3, when creating a new volume from a snapshot, your gateway keeps the snapshot data in Amazon S3 where it becomes the primary data for your new volume.
Q. Can I access my snapshots from within AWS?
Data written to your Gateway-Stored volumes is stored on your on-premises storage hardware, and asynchronously backed up to Amazon S3 in the form of Amazon EBS snapshots. You can use a snapshot of your Gateway-Stored volume as the starting point for a new Amazon EBS volume which you can then attach to an Amazon EC2 instance. This allows you to easily mirror data from your on-premises applications to your applications running on Amazon EC2 in the event you require additional on-demand compute capacity for data processing or replacement capacity for disaster recovery purposes.
Q. Are snapshots versioned? Can I read an older snapshot to do a point-in-time recovery?
Each snapshot is given a unique identifier which can be viewed using the AWS Management Console. You can create AWS Storage Gateway or Amazon EBS volumes based on any of your existing snapshots by specifying this unique identifier.
Q. Do the AWS Storage Gateway’s volumes need to be un-mounted in order to take a snapshot? Does the snapshot need to complete before the volume can be used again?
No, taking snapshots does not require you to un-mount your volumes, nor does it impact your application’s performance. However, snapshots only capture data that has been written to your AWS Storage Gateway volume, which may exclude any data that has been locally buffered by your application or OS.
Q. Can I schedule snapshots of my gateway’s storage volumes?
Yes, you can create a snapshot schedule for each of your storage volumes. You can modify both the time the snapshot occurs each day, as well as the frequency (every 1, 2, 4, 8, 12, or 24 hours).
Q. How long does it take to complete a snapshot?
You can take snapshots of your Gateway-Cached volume in Amazon S3, or your on-premises Gateway-Stored volume. These snapshots are stored as Amazon EBS snapshots in Amazon S3. The time it takes to complete a snapshot is largely dependent upon the size of your volume and the speed of your internet connection to AWS. The AWS Storage Gateway compresses all data prior to upload, reducing the time to take a snapshot.
Q. Will I be able to access my snapshot data or Gateway-Cached volume data using Amazon S3’s APIs?
No, snapshots are only accessible from the AWS Storage Gateway and Amazon EBS and cannot be directly accessed using Amazon S3 APIs. Similarly, volumes can only be accessed from the AWS Storage Gateway.
Monitoring and Maintenance
Q. How do I monitor my gateway?
You can use Amazon CloudWatch to monitor the performance metrics for your AWS Storage Gateway volumes, giving you insight into bandwidth, throughput, and latency. For your Gateway-Cached volumes, you can also monitor your CacheHitPercent to ensure your local cache storage is large enough to store your recently accessed data for low-latency on-premises access. These metrics are accessible from the AWS Management Console. You can refer to our technical documentation to learn more.
Q. Does the AWS Storage Gateway support bandwidth throttling?
Yes, using the AWS Management Console, you can restrict the bandwidth between your gateway and AWS based on a rate that you provide. You can specify individual rates for inbound and outbound traffic.
Q. How does the AWS Storage Gateway manage updates?
When configuring your gateway, you can specify a weekly maintenance schedule. This allows you to control when the AWS Storage Gateway service can deploy important updates and software patches to your local gateway. Updates should take only a few minutes to complete.
Q. Does AWS Premium Support cover the AWS Storage Gateway?
Yes, AWS Premium Support covers issues related to your use of the AWS Storage Gateway. Please see the AWS Premium Support detail page for further details and pricing.
Q: What other support options are available?
You can tap into the breadth of existing AWS community knowledge through the AWS Storage Gateway discussion forum.