Introduction

Amazon Web Services (AWS) offers a broad portfolio of reliable, scalable, and secure storage services for storing, accessing, protecting, and analyzing your data. This makes it easier to match your storage methods with your needs, and provides storage options that are not easily achievable with on-premises infrastructure. When selecting a storage service, ensuring that it aligns with your access patterns will be critical to achieving the performance you want.

You can select from block, file, and object storage services as well as cloud data migration options for your workload. Choosing the right storage service for your workload requires you to make a series of decisions based on your business needs.

This decision guide will help you ask the right questions, provide a clear path for implementation, and help you migrate from your existing on-premises storage.

This six minute clip is from a 55 minute recording of a presentation by AWS senior storage solutions architects Kevin McDonald and Victor Munoz at the 2022 AWS Summit. It provides an overview of available AWS storage services.

Time to read

20 minutes

Purpose

Help determine which AWS storage service is the best fit for your needs.

Level

Beginner

Last updated

June 2, 2023

Understand

Data is a cornerstone of successful application deployments, analytics workflows, and machine learning innovations. Well-architected systems use multiple storage services and enable different features to improve performance.

In many cases, however, choosing the right storage service will start with how well it aligns with what you're already using (or are familiar with). Working with storage services that you are familiar with will make it easier for you to get started - and can make migration of your data easier and potentially faster.

For example, services in the Amazon FSx data storage family come in four options that align to popular file systems:

  • Amazon FSx for Windows File Server provides fully managed Microsoft Windows file servers, backed by a fully native Windows file system.
  • Amazon FSx for Lustre allows you to launch and run the high-performance Lustre file system.
  • Amazon FSx for OpenZFS a fully managed file storage service that enables you to move data to AWS from on-premises ZFS or other Linux-based file servers.
  • Amazon FSx for NetApp ONTAP a fully managed service that provides highly reliable, scalable, high-performing, and feature-rich file storage built on NetApp's popular ONTAP file system.

Consider

You might be considering AWS storage services because you are migrating an existing application to the cloud or building a new application in the cloud. When moving data to the cloud, it is important for you to understand where you are moving it, the potential use cases, the type of data you are moving, and the network resources available.

Here's some of the criteria to consider when choosing an AWS storage service.   

  • AWS storage services offer multiple protocol options:

    • Block storage offers high-performance storage that is direct-attached to a compute instance with low-latency access, making it suitable for applications that require fast and consistent I/O operations.
    • File-based storage is natively mountable from virtually any operating system using industry-standard protocols like NFS and SMB. It provides simple storage for workloads that need access to shared data across multiple compute instances.
    • Object storage provides easy access to data through an application programming interface (API) over the internet and is well-suited to read-heavy workloads (such as streaming applications and services).

    Protocols play a crucial role when considering AWS storage services as they determine how data is accessed, transferred, and managed within the storage environment.

  • It's important to consider the operating system of the clients that will be accessing the data. Windows-based clients can use file-based storage options such as Amazon FSx for Windows File Server. It provides highly available storage to your Windows applications with full Server Message Block (SMB) support.

    Amazon FSx for Lustre (for high-performance file systems) is designed for use with Unix/Linux-based file systems. FSx for Lustre is optimized for workloads where speed matters, such as machine learning, high performance computing (HPC), video processing, and financial modeling.

    The choice of client type for an AWS storage service is critical to ensure easy access and sharing of data across workloads. Selecting a service that is compatible with the file systems and protocols used by your clients is key to avoiding compatibility issues and ensuring seamless data access and transfer.

  • Performance is a critical factor to consider when choosing an AWS storage service. There are several factors to consider when evaluating storage performance, including IOPS (input/output operations per second), access patterns, latency, and throughput or bandwidth. It is important to ask questions such as:

    • Is your workload latency sensitive?
    • Do other metrics (such as IOPS or throughput) dominate your applications performance profile?
    • Is your workload read or write-heavy?
  • Some factors to consider when migrating your on-premises storage to AWS are:

    • Data transfer: what is the most efficient method to transfer your data to AWS?
    • Compatibility: For example, if you already leverage NetApp ONTAP appliances on-premises services (such as Amazon FSx for NetApp ONTAP) provide a seamless migration path.
    • Application integration: Evaluate how your applications will integrate with AWS storage services. Consider any necessary modifications or configurations required to enable seamless connectivity and functionality between your applications and the AWS environment.
    • Data Management and lifecycle: Plan for data management tasks such as backup, replication, and lifecycle management in the AWS environment. Consider AWS services and features that can help automate these tasks, such as versioning, lifecycle policies, and cross-region replication.
    • Security and compliance: Ensure that your data remains secure during the migration process. Implement appropriate security measures, such as encryption and access controls, to protect your data both in transit and at rest
    • Cost optimization: Analyze the cost implications of migrating your storage solution to AWS. Consider factors such as storage pricing, data transfer costs, and any associated services or features required to optimize costs.

    By carefully considering these factors, you can ensure a successful migration from an on-premises storage solution to AWS storage services, minimizing disruptions, and maximizing the benefits of cloud storage.

  • Backup and protection requirements are critical factors to consider when choosing an AWS storage service because they help ensure the availability and durability of your data.

    Without adequate backup and protection measures, data can be lost due to accidental deletion, hardware failure, or natural disasters, which can have severe consequences for your business.

    Familiarize yourself with services such as AWS Backup, which can backup your data on demand or automatically as part of a scheduled backup plan. AWS Backup also offers cross-region replication which can be particularly valuable if you have business continuity or compliance requirements to store backups a minimum distance away from your production data.

  • Disaster recovery is a critical consideration when choosing an AWS storage service because it helps ensure business continuity in the event of a disaster or outage. A disaster can be caused by various factors, such as natural disasters, human error, or cyberattacks, and can result in significant data loss and downtime.

    Choosing a storage service that provides disaster recovery features, such as replication across multiple availability zones, can help minimize the impact of a disaster on your business. It's important to consider factors such as recovery time objectives (RTO) and recovery point objectives (RPO) when evaluating disaster recovery options and choose a storage service that meets your business needs.

  • Beyond the base storage costs, there are other factors that impact pricing such as storage capacity, data transfer, and availability that impacts the total cost of storage. The following can help you reduce cost when using an AWS storage service:

    • Use the appropriate storage service for your workload type
    • Use AWS Cost Explorer and other billing tools to monitor organizational spend
    • Understand your data and how it is being used

    We also recommend that you use the AWS Pricing Calculator to estimate your cost when choosing an AWS storage service.

  • Security at AWS is a shared responsibility. AWS provides a secure foundation for customers to build and deploy their applications, but customers are responsible for implementing their own security measures to protect their data, applications, and infrastructure.

    You should consider aspects of security such as access control, data encryption, compliance requirements, monitoring and logging, and incident response when choosing an AWS storage service. By doing so, you can help ensure that your data is protected while using AWS services.

Choose

Now that you know the criteria you should use to evaluate your storage options, you are ready to choose which AWS storage services are right for your business needs.

The following table highlights which storage options are optimized for which circumstances. Use it to help determine the one that is the best fit for your use case.

Storage type
What is it optimized for?
Storage services or tools
Close

Block storage is technology that controls data storage and storage devices. It takes any data, like a file or database entry, and divides it into blocks of equal sizes. The block storage system then stores the data block on underlying physical storage in a manner that is optimized for fast access and retrieval.

Applications requiring low-latency, high-performance durable storage attached to single EC2 instances or containers, such as databases and general-purpose local instance storage.
Close

Amazon EBS

Amazon Elastic Block Store (Amazon EBS) provides block level storage volumes for use with EC2 instances. EBS volumes behave like raw, unformatted block devices. You can mount these volumes as devices on your instances. EBS volumes that are attached to an instance are exposed as storage volumes that persist independently from the life of the instance.

Close

Amazon EC2 instance store

An instance store provides temporary block-level storage for your instance. This storage is located on disks that are physically attached to the host computer. Instance store is ideal for temporary storage of information that changes frequently, such as buffers, caches, scratch data, and other temporary content. It can also be used to store temporary data that you replicate across a fleet of instances, such as a load-balanced pool of web servers.

Close

File systems store data in a hierarchical structure of files and folders. In network environments, file-based storage often uses network-attached storage (NAS) technology. NAS allows users to access network storage data in similar ways to a local hard drive. File storage is user-friendly and allows users to manage file-sharing control.

Applications and workloads requiring shared read and write access across multiple EC2 instances/containers or from multiple on-prem servers, such as team file shares, highly-available enterprise applications, analytics workloads, and ML training.
Close

Amazon Elastic FIle System

Amazon EFS provides serverless, fully elastic file storage so that you can share file data without provisioning or managing storage capacity and performance. Amazon EFS is built to scale on demand to petabytes without disrupting applications, growing and shrinking automatically as you add and remove files.

Close

Amazon FSx

Amazon FSx is designed to provide an easy and cost effective way to launch, run, and scale feature-rich, high-performance file systems in the cloud. It supports a range of workloads. With Amazon FSx, you can choose between four widely-used file systems: Lustre, NetApp ONTAP, OpenZFS, and Windows File Server.

Close

Amazon FSx for Lustre

Amazon FSx for Lustre is designed for high performance computing scenarios using the open-source Lustre file system. It eliminates the traditional complexity of setting up and managing Lustre file systems, enabling you to spin up and run high-performance file system quickly. It also provides multiple deployment options so you can optimize cost for your needs.

Close

Amazon FSx for NetApp ONTAP

Amazon FSx for ONTAP provides feature-rich, fast, and flexible shared file storage that’s broadly
accessible from Linux, Windows, and macOS compute instances running in AWS or on premises. It offers high-performance solid state drive (SSD) storage with submillisecond
latencies, allowing you to achieve SSD levels of performance for your workload while paying for SSD storage for only a small fraction of your data.

Close

Amazon FSx for OpenZFS

Amazon FSx for OpenZFS is a fully managed file storage service that makes it easy to move data residing in on-premises ZFS or other Linux-based file servers to AWS without changing your application code or how you manage data. It offers highly reliable, scalable, performance, and feature-rich file storage built on the open-source OpenZFS file system, providing the familiar features and capabilities of OpenZFS file systems with the agility, scalability, and simplicity of a fully managed AWS service.

Close

Amazon FSx for Windows File Server

Amazon FSx for Windows File Server provides fully managed Microsoft Windows file servers, backed by a fully native Windows file system. It has the features, performance, and compatibility to easily lift and shift enterprise applications to the AWS Cloud.Amazon FSx supports a broad set of enterprise Windows workloads with fully managed file storage built on Microsoft Windows Server. Amazon FSx has native support for Windows file system features and for the industry-standard Server Message Block (SMB) protocol to access file storage over a network.

Close

AWS Storage Gateway

AWS Storage Gateway is a service that connects an on-premises software appliance with cloud-based storage to provide seamless and secure integration between your on-premises IT environment and the AWS storage infrastructure in the AWS Cloud.

Close

Object storage is a technology that stores and manages data in an unstructured format called objects. Each object is tagged with a unique identifier and contains metadata that describes the underlying content.

Read-heavy workloads such as content distribution, web hosting, big data analytics, and ML workflows. Well-suited for scenarios where data needs to be stored, accessed, and distributed globally over the internet.
Close

Amazon S3

Amazon Simple Storage Service (Amazon S3) is an object storage service that offers scalability, data availability, security, and performance. Amazon S3 provides management features so that you can optimize, organize, and configure access to your data to meet your specific business, organizational, and compliance requirements.

Close

A cache is a high-speed data storage layer used to temporarily store frequently accessed or recently used data closer to the point of access, with the aim of improving system performance and reducing latency. It serves as a buffer between the slower and larger primary storage (such as disks or remote storage) and the computing resources that need to access the data.

Fully managed, scalable, and high-speed cache on AWS for processing file data stored in disparate locations—including on premises NFS file systems, and/or in cloud file systems (Amazon FSx for OpenZFS, Amazon FSx for NetApp ONTAP), and Amazon S3.
Close

Amazon File Cache

Amazon File Cache is a fully managed, high-speed cache on AWS that's used to process file data, regardless of where the data is stored. It serves as a temporary, high-performance storage location for data that is stored in on-premises file systems, AWS FSx, and Amazon S3 buckets.

Close

AWS Storage Gateway

AWS Storage Gateway is a service that connects an on-premises software appliance with cloud-based storage to provide seamless and secure integration between your on-premises IT environment and the AWS storage infrastructure in the AWS Cloud.

In addition to choosing a storage service, you will need to make choices about how you migrate your data to live within the chosen services. AWS offers several choices to migrate your data - based on whether it needs to live online or offline. There are two key considerations:

  • Speed -  Choose online migration when speed matters. Online is measured in minutes or hours vs. offline (where it can be measured by days). If data is frequently updated and time-critical, choose online. Choose offline when it’s a one-time move, and not time-critical.
  • Bandwidth - Moving data online takes away from available bandwidth used for day-to-day. Choose offline when there are network constraints, and data can be offline while in transit without disrupting your business. AWS services in the Snow Family offer an option for offline migration. 

The following table provides a detailed look at your online/offline options.

Migration options
When speed is the priority
When bandwidth is important
Storage services or tools
Close

Online migration involves transferring data and applications over the internet while they are still running in the on-premises data center. This approach can be more efficient than offline migration since it minimizes downtime and enables organizations to start using cloud resources sooner. However, it requires a reliable internet connection and may not be suitable for large amounts of data or mission-critical applications.

Online is optimized for frequent updates to data. Use it for time-critical or ongoing workloads.
You need to ensure that you have sufficient bandwidth available, but can gain benefit by optimizing the transfer time of day.
Close

AWS DataSync

AWS DataSync is an online data movement and discovery service that simplifies data migration and helps you quickly, easily, and securely move your file or object data to, from, and between AWS storage services.

Close

AWS Transfer Family

AWS Transfer Family is a secure transfer service that stores your data in Amazon Simple Storage Service or Amazon Elastic File System and simplifies the migration of Secure File Transfer Protocol (SFTP), File Transfer Protocol Secure (FTPS), File Transfer Protocol (FTP), and Applicability Statement 2 (AS2) workflows to AWS.

Close

Amazon FSx for NetApp ONTAP SnapMirror

It replicates data at high speeds, so you get high data availability and fast data replication across ONTAP systems, whether you're replicating between two Amazon FSx file systems in AWS, or from on-premises to AWS.

Close

Offline migration involves moving data and applications without any connection to the internet. This approach requires physically transporting the data on external hard drives or other storage media to the cloud provider’s data center. This method is typically used when there are large amounts of data to transfer, limited bandwidth or connectivity, or concerns about security and privacy.

Suitable for one-time or periodic uploads - and when data can be static in transit.
This choice makes sense when you need to use only the minimum available bandwidth - and you prefer the predictability of physical moves.
Close

AWS Snowball

The AWS Snowball service uses physical storage devices to transfer large amounts of data between Amazon S3 and your onsite data storage location at faster-than-internet speeds.

Close

AWS Snowcone

AWS Snowcone is a portable, rugged, and secure device for edge computing and data transfer. You can use a Snowcone device to collect, process, and move data to the AWS Cloud, either offline by shipping the device to AWS, or online by using AWS DataSync.

Use

Now that you have determined the best protocol you need to work with your data, your performance requirements, and other criteria discussed in this guide, you should also have an understanding of which storage service would be the best fit for your needs.  

To explore how to use and learn more about each of the available AWS storage services - we have provided a pathway to explore how each of the services work. The following section provides links to in-depth documentation, hands-on tutorials, and resources to get you started.

  • Amazon S3
  • Amazon S3

    Getting started with Amazon S3

    This guide will help you get started with Amazon S3 by working with buckets and objects. A bucket is a container for objects. An object is a file and any metadata that describes that file.

    Amazon S3

    Optimizing Amazon S3 Performance 

    When building applications that upload and retrieve storage from Amazon S3, follow the AWS best practices guidelines in this paper to optimize performance.
     

    Amazon S3

    Amazon S3 tutorials
     

    The following tutorials present complete end-to-end procedures for common Amazon S3 tasks. These tutorials are intended for a lab-type environment and provide general guidance.

  • Amazon EBS
  • Amazon EBS

    Getting started with Amazon EBS



    Amazon EBS is recommended for data that must be quickly accessible and requires long-term persistence.
     
    Amazon EBS

    Create an Amazon EBS volume



    An Amazon EBS volume is a durable, block-level storage device that you can attach to your instances.

    Get started with the tutorial »

    Amazon EBS

    Use Amazon EBS direct APIs to access the contents of an EBS snapshot
     

    You can use the direct APIs to create EBS snapshots, write and read data on your snapshots, and identify differences.

    Explore the guide »

  • Amazon EFS
  • AWS Database Migration Service

    Getting started with Amazon EFS


    Learn how to create an Amazon EFS file system. You will mount your file system on an Amazon EC2 instance in your VPC, and test the end-to-end setup.

    Get started with the tutorial »

    AWS Database Migration Service

    Create a Network File System


    Learn how to store files and create an Amazon EFS file system, launch a Linux virtual machine on Amazon EC2, mount the file system, create a file, terminate the instance, and delete the file system.

    Get started with the tutorial »

    AWS Database Migration Service

    Set up an Apache web server and serve Amazon EFS files

    Learn how to set up an Apache web server on an EC2 instance and set up an Apache web server on multiple EC2 instances by creating an Auto Scaling group.
     

    Get started with the tutorial »

  • Amazon FSx
  • AWS Database Migration Service

    Getting started with Amazon FSx
     


    This getting started guide walks you through what you'll need to do to begin using Amazon FSx.


    Explore the guide »

    AWS Database Migration Service

    Getting started with Amazon FSx for Lustre
     

    Learn how to use your Amazon FSx for Lustre file system to process the data in your Amazon S3 bucket with your file-based applications.

    Explore the guide »

    AWS Database Migration Service

    What is FSx for Windows File Server?

    This guide provides an introduction to Amazon FSx for Windows File Server


    Explore the guide »

    AWS Database Migration Service

    Getting started with Amazon FSx for NetApp ONTAP
     

    Learn how to get started using Amazon FSx for NetApp ONTAP.



    Get started with the tutorial »

    AWS Database Migration Service

    Getting started with Amazon FSx for OpenZFS
     

    Learn how to create an Amazon FSx for Lustre file system and access it from your compute instances.

    Get started with the tutorial »

  • Amazon File Cache
  • AWS File Cache

    Getting started with Amazon File Cache

    Learn how to create an Amazon File Cache resource and access it from your compute instances.

    Get started with the tutorial »

    Amazon File Cache

    Amazon File Cache in action

    This video shows how Amazon File Cache can be used as a temporary high performance storage location for data stored in on premises file systems.

    Watch the video »

  • AWS Storage Gateway
  • AWS Database Migration Service

    Getting started with AWS Storage Gateway

    Connect an on-premises software appliance with cloud-based storage to provide seamless and secure integration. Find documentation content by gateway type.


    Explore the guides »

    AWS Database Migration Service

    User Guide for Amazon S3 File Gateway

    Describes Amazon S3 File Gateway concepts and provides instructions on using the various features with both the console and the API.



    Explore the guide »

    AWS Database Migration Service

    User Guide for Amazon FSx File Gateway

    Describes Amazon FSx File Gateway, which provides access to in-cloud Amazon FSx for Windows File Server shares from on-premises facilities. Includes instructions on working with the console and the API.
    Explore the guide »

  • AWS DataSync
  • AWS Database Migration Service

    Getting started with AWS DataSync

    This guide walks through how you can get started with AWS DataSync by using the AWS Management Console.


    Explore the guide »

    AWS Database Migration Service

    Using AWS DataSync to move data from Hadoop to Amazon S3

    Understand how you can accelerate your data migrations and simplify data transfers between their cluster and AWS.


    Read the blog »

    AWS Database Migration Service

    AWS DataSync tutorials


    These tutorials walk you through some real-world scenarios with AWS DataSync and transferring data.


    Get started with the tutorials »

  • AWS Transfer Family
  • Amazon Aurora

    Getting started with AWS Transfer Family

    Learn how to create an SFTP-enabled server with publicly accessible endpoint using Amazon S3 storage, add a user with service-managed authentication, and transfer a file with Cyberduck.

    Get started with the tutorial »

    Amazon Aurora

    AWS Transfer Family in action


    This video shows how the AWS Transfer Family can be used for each of the three supported protocols (SFTP, FTPS, and FTP), both over the public internet, as well as within a VPC.

    Watch the video »

  • AWS Snow Family
  • AWS Database Migration Service

    Getting started with AWS Snow Family

    These guides provide links to documentation covering all current services in the Snow Family.


    Read the documentation »

    AWS Database Migration Service

    AWS Snowcone User Guide


    This guide includes guidance for storage and local compute, importing and exporting data into the AWS Cloud, and other features of a Snowcone device.

    Explore the guide »

    AWS Database Migration Service

    AWS Snowball Edge Developer Guide

    This guide includes guidance for local storage and compute, clustering, importing and exporting data into Amazon S3,  and other features of an Edge device.  

    Explore the guide »

Explore

Architecture diagrams

Explore reference architecture diagrams to help you develop, scale, and test storage services on AWS.

Explore architecture diagrams »

Whitepapers

Explore whitepapers to help you get started, learn best practices, and understand your storage options.

Explore whitepapers »

AWS solutions

Explore vetted solutions and architectural guidance for common use cases for storage services.

Explore solutions »

Additional resources

Was this page helpful?