Deploying AWS Storage Gateway on Linux KVM hypervisor

AWS customers support a variety of on-premises environments and have different deployment needs. Until now, customers using Linux Kernel-based Virtual Machine (KVM) hypervisor have not had a convenient way to implement a hybrid cloud storage solution using AWS Storage Gateway. AWS Storage Gateway provides on-premises applications access to virtually unlimited cloud storage using NFS, SMB, iSCSI, and iSCSI-VTL interfaces. Customers use the service to back up and archive data to AWS, shift on-premises storage to cloud-backed file shares, and provide on-premises applications low-latency access to data in AWS.

Today we are making Storage Gateway available on Linux Kernel-based Virtual Machine (KVM) hypervisor, helping meet the needs of customers who use this cost-effective hypervisor. KVM is an open source virtualization technology that lets customers turn a Linux-based host machine into one or more virtual machines. KVM is included in all versions of Linux 2.6.20 and newer. Storage Gateway now supports QEMU-KVM on Red Hat Enterprise Linux 7.7, CentOS 7.7, and Ubuntu 18.04 LTS and 16.04 LTS, on host machines using Intel x86 CPUs with virtualization extension.

Storage Gateway supports KVM running on multiple Linux distributions, and there are multiple tools you can use to manage KVM virtual machines. In this blog post, we focus on how to install Storage Gateway on KVM hypervisor on CentOS 7.7 using KVM virsh. While we used CentOS 7.7, the same instructions also apply to other supported Linux distributions. KVM virsh is a command line interface tool that you can use to manage virtual machines. It appeals to advanced Linux administrators interested in automating virtual machine management through scripts. You can also manage KVM virtual machines through a graphical user interface called Virtual Manager (virt-manager) or oVirt, an open-source virtualization management tool.

Deploying Storage Gateway on KVM

To get started with using Storage Gateway on KVM, log into your AWS Management Console and search for Storage Gateway. Then, click on the Get Started button as shown in the following screenshot.

From there, create a gateway and Select gateway type you would like to launch (File, Volume, or Tape) on your virtual machine.

Next, you see the Select host platform screen. Choose the Linux KVM option and click on the Download image button to download the KVM image for Storage Gateway.

Once you have downloaded the image, you unzip it to extract Storage Gateway’s KVM image in qcow2 format. The libvirt domain XML file contains the default VM configuration for Storage Gateway. Based on your needs, you can modify this file to allocate appropriate VM resources for Storage Gateway.

You can use the domain XML file to define the Storage Gateway VM using the following command, or you can manually install the VM with your own parameters and settings.

virsh define aws-storage-gateway-<providednumber>.libvirt.domain.xml

The following screenshot shows how you can use the included domain XML file to define your Storage Gateway VM.

Alternately, the following command and screenshot show how to manually install the Storage Gateway VM with your own parameters and settings, such as different memory or vCPU allocation.

KVM command:

virt-install \
--name "SGW_KVM" \
--description "SGW KVM" \
--os-type=generic \
--ram=32768 \
--vcpus=16 \
--disk path=kvm.qcow2,bus=virtio,size=80 \
--disk path=kvm-cache.qcow2,bus=virtio,size=160 \
--disk path=kvm-upload.qcow2,bus=virtio,size=160 \
--network default,model=virtio \
--graphics none \
--import

Using either method, you can define your Storage Gateway VM and start it. You can find the operating state of your VM using the following command:

virsh list -all 

Once your gateway VM is running, you can log into the gateway using the following command and passing the VM name in the command:

virsh console

Log in using the default user name and password provided in the Storage Gateway User Guide. Afterward, you can change your network settings, modify your gateway parameters, and test network connectivity to AWS through the gateway local console as shown in the following screenshot.

To leave the local console, you can press CTRL and ], which drops you back to your Linux shell.

Next, you can enter the VM’s IP address in the AWS Storage Gateway console and follow the steps described here to activate your gateway. These follow-on steps are the same for each host platform type.

Summary

Today’s launch of AWS Storage Gateway compatibility with Linux KVM hypervisor continues the service’s mission to make it convenient for customers to deploy Storage Gateway in their environment without requiring changes to their existing infrastructure. AWS Storage Gateway launched with the ability to deploy a local gateway appliance on a VMware ESXi virtual machine and quickly added support for Amazon EC2. Later, we expanded to include Microsoft Hyper-V hypervisor support, and in September 2018, we made Storage Gateway available on a hardware appliance. In November 2019, we announced High Availability support for Storage Gateways running on VMware vSphere clusters, helping customers automatically recover from most service interruptions in under 60 seconds. We look forward to seeing where Storage Gateway takes our customers in the future.

In this post, we showed how you can deploy Storage Gateway on the KVM hypervisor. If you have chosen Linux KVM as the virtualization platform of choice for your on-premises infrastructure, you now have the option to deploy Storage Gateway on KVM in your environment. This then enables you to start consuming AWS Cloud Storage services through the Storage Gateway. To learn more, see the Storage Gateway User Guide. Storage Gateway is now available in 20 AWS Regions, including AWS GovCloud (US-West) and China (Beijing). If you have any suggestions or questions, please comment on this post.