Getting Started with AWS Storage Gateway

You can begin using the AWS Storage Gateway in just a few steps. To get started, you simply:

• Download the AWS Storage Gateway virtual machine (VM) image from the AWS Management Console.
• Select between a Gateway-Cached or Gateway-Stored configuration. With Gateway-Cached volumes, you store your primary data in Amazon S3, and retain frequently accessed data locally. With Gateway-Stored volumes, you store your primary data locally, and upload backups to Amazon S3.
• Allocate local storage to your installed on-premises gateway from Direct Attached Storage (DAS), Network Attached Storage (NAS), or Storage Area Network (SAN) storage. For Gateway-Cached configurations, this local storage will be used for your frequently accessed data. For Gateway-Stored configurations, it will be used for your primary data.
• Activate your on-premises gateway by using the AWS Management Console to associate your gateway’s IP Address with your AWS Account and select an AWS Region for your gateway to store uploaded data.
• Use the AWS Management Console to create AWS Storage Gateway volumes and attach these volumes as iSCSI devices to your on-premises application servers.

By following these steps, you can begin using your existing on-premises applications to seamlessly store data in Amazon S3. These applications can now write data to their attached AWS Storage Gateway volumes. Your application data will either be stored directly in Amazon S3 for substantial cost savings on primary storage (Gateway-Cached volumes), or will be stored locally and backed up to Amazon S3 for durable and cost-effective backups (Gateway-Stored volumes).

Service Highlights

Secure – The AWS Storage Gateway securely transfers your data to AWS over SSL and stores data encrypted at rest in Amazon S3 using Advanced Encryption Standard (AES) 256, a secure symmetric-key encryption standard using 256-bit encryption keys.

Durably backed by Amazon S3 – The AWS Storage Gateway durably stores your on-premises application data by uploading it to Amazon S3. Amazon S3 stores data in multiple facilities and on multiple devices within each facility. Amazon S3 also performs regular, systematic data integrity checks and is built to be automatically self-healing.

Compatible – There is no need to re-architect your on-premises applications. The AWS Storage Gateway exposes a standard iSCSI interface that works with your existing applications.

Cost-Effective – By making it easy for your on-premises applications to store data on Amazon S3, the AWS Storage Gateway reduces the cost, maintenance, and scaling challenges associated with managing primary and backup storage environments. You pay only for what you use with no long-term commitments.

Designed for use with other Amazon Web Services – The AWS Storage Gateway is designed to seamlessly integrate with Amazon S3, Amazon EBS, and Amazon EC2 by enabling you to store point-in-time snapshots of your on-premises application data in Amazon S3 as Amazon EBS snapshots for future recovery on-premises or in Amazon EC2. This integration allows you to easily mirror data from your on-premises applications to applications running on Amazon EC2 in disaster recovery (DR) and on-demand compute capacity cases.

Optimized for Network Efficiency – The AWS Storage Gateway efficiently uses your internet bandwidth to speed up the upload of your on-premises application data to AWS. The AWS Storage Gateway only uploads data that has changed, minimizing the amount of data sent over the internet. You can also use AWS Direct Connect to further increase throughput and reduce your network costs by establishing a dedicated network connection between your on-premises gateway and AWS.

Features

• Gateway-Cached Volumes – Gateway-Cached volumes allow you to utilize Amazon S3 for your primary data, while retaining some portion of it locally in a cache for frequently accessed data. These volumes minimize the need to scale your on-premises storage infrastructure, while still providing your applications with low-latency access to their frequently accessed data. You can create storage volumes up to 32 TBs in size and mount them as iSCSI devices from your on-premises application servers. Data written to these volumes is stored in Amazon S3, with only a cache of recently written and recently read data stored locally on your on-premises storage hardware.

• Gateway-Stored Volumes – Gateway-Stored volumes store your primary data locally, while asynchronously backing up that data to AWS. These volumes provide your on-premises applications with low-latency access to their entire data sets, while providing durable, off-site backups. You can create storage volumes up to 1 TB in size and mount them as iSCSI devices from your on-premises application servers. Data written to your Gateway-Stored volumes is stored on your on-premises storage hardware, and asynchronously backed up to Amazon S3 in the form of Amazon EBS snapshots.

• Data Snapshots – The AWS Storage Gateway provides the ability to create and store point-in-time snapshots of your storage volumes in Amazon S3. These snapshots are stored as Amazon EBS snapshots. You can set the schedule for when snapshots occur or create them ad-hoc. Snapshots are incremental backups, reducing your storage charges. When taking a new snapshot, only the data that has changed since your last snapshot is stored. All snapshot storage is also compressed, further minimizing your storage charges.

• Metrics – You can use Amazon CloudWatch to monitor the performance metrics of your on-premises gateway. This provides insight into your on-premises applications’ throughput, latency, and internet bandwidth to Amazon S3. These metrics are accessible from the AWS Management Console.

Pricing

Gateway Pricing

• $125.00 per activated virtual gateway appliance / month

Storage Pricing

You are billed for the Gateway-Cached volume data that you store in Amazon S3. You only pay for what you use.

You are billed for your storage volume snapshots stored in Amazon S3. These snapshots are stored and billed as Amazon EBS snapshots.

Data Transfer Pricing

The pricing below is based on data transferred “in” and “out” of AWS by the AWS Storage Gateway.

Rate tiers take into account your aggregate Data Transfer Out usage across AWS Storage Gateway, Amazon EC2, Amazon EBS, Amazon S3, Amazon Glacier, Amazon RDS, Amazon SimpleDB, Amazon SQS, Amazon SNS, Amazon DynamoDB, and Amazon VPC.

Common Use Cases

The AWS Storage Gateway can be used to support a wide variety of use cases, including:

Corporate File Sharing

Managing on-premises storage for departmental file shares and home directories typically results in high capital and maintenance costs, under-utilized hardware, and restrictive user quotas. The AWS Storage Gateway addresses these on-premises scaling and maintenance issues by enabling you to seamlessly store your corporate file shares on Amazon S3, while keeping a copy of your frequently accessed files on-premises. This minimizes the need to scale your on-premises file storage infrastructure, while still providing low-latency access to your frequently accessed data. Using the AWS Storage Gateway, you can create Gateway-Cached storage volumes up to 32 TBs in size and mount them as iSCSI devices from your on-premises file servers. You can then expose these volumes as Common Internet File System (CIFS) shares or Network File System (NFS) mount points to your client machines. The AWS Storage Gateway durably stores files written to these shares or mount points in Amazon S3, while maintaining a cache of recently written and recently read files locally on your on-premises storage hardware for low-latency access. Since you only pay for the storage you actually use, you can scale your storage on-demand and avoid the costs of under-utilized hardware.

Backup

The AWS Storage Gateway enables your existing on-premises backup applications to store primary backups on Amazon S3’s scalable, reliable, secure, and cost-effective storage service. You can create Gateway-Cached storage volumes and mount them as iSCSI devices to your on-premises backup application servers. All data is securely transferred to AWS over SSL and stored encrypted in Amazon S3 using AES 256-bit encryption. Using Gateway-Cached volumes provides an attractive alternative to the traditional choice of maintaining and scaling costly storage hardware on-premises.

For scenarios where you want to keep your primary data or backups on-premises, you can use Gateway-Stored volumes to keep this data locally, and backup this data off-site to Amazon S3. Gateway-Stored volumes provide an attractive alternative to dealing with the longer recovery times and operational burden of managing off-site tape storage for backups.

Disaster Recovery and Resilience

The AWS Storage Gateway, together with EC2, can mirror your entire production environment for disaster recovery (DR). Planning for business continuity in the event of a power outage, fire, flood, or other disaster can be challenging. It requires investments in redundant infrastructure and staff across multiple datacenters and costly storage replication solutions. AWS Storage Gateway and Amazon EC2 together provide a simple cloud-hosted DR solution. Using Amazon EC2, you can configure virtual machine images of your DR application servers and only pay for these servers when you need them. In the event your on-premises infrastructure goes down, you simply launch the Amazon EC2 compute instances you need and attach them to copies of your on-premises data. The AWS Storage Gateway addresses the challenges of replicating data for DR by enabling you to create Gateway-Cached volumes that store your data in Amazon S3. By storing your data using the AWS Storage Gateway, you will be prepared for DR if you lose your on-premises application or storage.

Data Mirroring to Cloud-Based Compute Resources

If you want to leverage Amazon EC2’s on-demand compute capacity for additional capacity during peak periods, for new projects, or as a more cost-effective way to run your normal workloads, you can use the AWS Storage Gateway to mirror your volume data to Amazon EC2 instances. If you’re running development and User Acceptance Testing (UAT) environments in Amazon EC2 to take advantage of AWS’s on-demand compute capacity, you can use the AWS Storage Gateway to ensure these environments have ongoing access to the latest data from your production systems on-premises.

Resources

Detailed Description

Hardware Requirements

The AWS Storage Gateway’s virtual machine (VM) must be installed on a host with the following minimum requirements:

• VMWare ESXi Hypervisor (v4.1 or v5) or Microsoft Hyper-V Server 2008 R2.

• 4 Virtual Processors assigned to the VM.

• 7.5GB of RAM assigned to the VM.

• 75GB of disk space for installation of virtual machine image and system data.

The AWS Storage Gateway currently supports mounting of its storage volumes using either the Microsoft Windows or Red Hat iSCSI Software Initiators.

How AWS Storage Gateway Works

The following diagram provides an overview of the AWS Storage Gateway deployment:

Once you’ve completed the setup process, you’ll have installed the AWS Storage Gateway’s virtual appliance on a host in your datacenter, specified whether you want to use Gateway-Cached or Gateway-Stored volumes, selected an AWS Region to store your data, and activated your gateway by associating its IP Address with your AWS Account. You can then create storage volumes and mount these storage volumes to your on-premises application servers as iSCSI devices.

Gateway-Cached volumes allow you to utilize Amazon S3 for your primary data, while retaining some portion of it locally in a cache for frequently accessed data. As your applications write data to and read data from a Gateway-Cached volume, this data is initially stored on-premises on Direct Attached Storage (DAS), Network Attached Storage (NAS), or Storage Area Network (SAN) storage. This local storage is used for two purposes. First, this local storage is used to prepare and buffer data for upload to your storage volume in Amazon S3. Second, this local storage is used to cache your application’s recently written and recently read data on-premises for low-latency access. When your application reads data from your Gateway-Cached volume, your on-premises gateway first checks its local cache for this data before checking Amazon S3.

Gateway-Stored volumes store your primary data locally, while asynchronously backing up that data to AWS. Your Gateway-Stored volumes are mapped to on-premises DAS, NAS, or SAN storage. You can start with either new storage or storage already holding data. As your on-premises applications write data to and read data from your storage volume, this data is retrieved locally from or stored locally on the on-premises DAS, NAS, or SAN storage you mapped to your storage volume. Your on-premises gateway also temporarily stores this data on local DAS, NAS, or SAN storage to prepare and buffer it for upload to Amazon S3, where it is stored in the form of Amazon EBS snapshots.

Whether you’re using Gateway-Cached or Gateway-Stored volumes, you can take point-in-time, incremental snapshots of your Storage Gateway volume and store them in Amazon S3 in the form of Amazon EBS snapshots. For Gateway-Stored volumes, where your volume data is stored on-premises, snapshots provide durable, off-site backups in Amazon S3. In the event you need to recover a backup, you can create a new Gateway-Stored volume from a snapshot stored in Amazon S3. Because Gateway-Stored volumes store your primary data locally, when creating a new volume from a snapshot, your on-premises gateway downloads the data contained within the snapshot to your local hardware, where it becomes the primary data for your new volume. You can also use a snapshot as the starting point for a new Amazon EBS volume which you can then attach to an Amazon EC2 instance.

For Gateway-Cached volumes, where your volume data is already stored in Amazon S3, snapshots can be used to preserve versions of your data. You can create a new Gateway-Cached volume from a snapshot stored in Amazon S3 in the event you need to recover a prior version. Because Gateway-Cached volumes store your primary data in Amazon S3, when creating a new volume from a snapshot, your on-premises gateway keeps the snapshot data in Amazon S3 where it becomes the primary data for your new volume.

Snapshots can be initiated on a scheduled or ad-hoc basis. When taking a new snapshot, only the data that has changed since your last snapshot is stored. If you have a volume with 100 GBs of data, but only 5 GBs of data have changed since your last snapshot, only the 5 additional GBs of snapshot data will be stored in Amazon S3. When you delete a snapshot, only the data not needed for any other snapshot is removed.

Getting Started Tutorial

Watch this video to learn how you can get started today using the AWS Storage Gateway.

Intended Usage and Restrictions

Your use of this service is subject to the Amazon Web Services Customer Agreement.