Cloud file storage is a method for storing data in the cloud that provides servers and applications access to data through shared file systems. This compatibility makes cloud file storage ideal for workloads that rely on shared file systems and provides simple integration without code changes.
A file system in the cloud is a hierarchical storage system that provides shared access to file data. Users can create, delete, modify, read, and write files and can organize them logically in directory trees for intuitive access.
Cloud file sharing can be defined as a service that provides simultaneous access for multiple users to a common set of file data in the cloud. Security for file sharing in the cloud is managed with user and group permissions enabling administrators to tightly control access to the shared file data.
Storing file data in the cloud delivers advantages in three key areas:
1. Scalability - Although not every cloud file storage solution can scale, leveraging all the capabilities of the cloud, the most advanced solutions provide the ability to start with the capacity and performance you need today and grow your capacity as needed. No more over provisioning to try and anticipate future needs.
2. Interoperability - Many existing applications require integration with shared file services that follow existing file system semantics. Cloud file storage solutions offer a distinct advantage as there is no new code to write to have secure, shared file access.
3. Budget and Resources - Operating file services on-premises requires budget for hardware, ongoing maintenance, power, cooling, and physical space. Cloud file storage enables organizations to redeploy technical resources to other projects that bring more value to the business.
Use cases such as large content repositories, development environments, media stores, and user home directories are ideal workloads for cloud file storage.
The flexibility to support and easily integrate with existing applications and the ease to deploy, manage, and maintain a file storage solution in the cloud provides the ability to support a broad spectrum of applications and verticals.
The need for shared file storage for web serving applications can be a challenge when integrating backend applications. Typically, there are multiple web servers delivering a web site’s content, with each web server needing access to the same set of files. Since cloud file storage solutions adhere to the expected file system semantics, file naming conventions, and permissions that web developers are accustomed to, cloud file storage can easily integrate their web applications.
Content Management Systems (CMS) require a common namespace and access to a file system hierarchy. Similar to web serving use cases, CMS environments typically have multiple servers that need access to the same set of files to serve up content. Since cloud file storage solutions adhere to the expected file system semantics, file naming conventions, and permissions that developers are accustomed to, storage of documents and other files can be easily integrated into existing CMS workflows.
Big data requires storage that can handle very large amounts of data while scaling to keep up with growth and providing the performance necessary to deliver data to analytics tools. Many analytics workloads interact with data via a file interface, rely on file semantics such as file locks, and require the ability to write to portions of a file. Since cloud file storage supports the needed file systems semantics and can scale capacity as well performance, it is ideal for delivering a file sharing solution that is easy to integrate into existing big data workflows.
Digital Media & Entertainment workflows are constantly changing. Many leverage a hybrid cloud deployment and need standardized access using network file protocols, such as NFS. These workflows require flexible, consistent, and secure access to data from off-the-shelf, custom-built, and partner solutions. Since cloud file storage adheres to existing file system semantics, storage of rich media content for processing and collaboration can be easily integrated for content production, digital supply chains, media streaming, broadcast playout, analytics and archive.
The use of home directories for storing files only accessible by specific users and groups is useful for many cloud workflows. Businesses that are looking to take advantage of the scalability and cost benefits of the cloud are extending access to home directories for many of their users. Since cloud file storage solutions adhere to required file system semantics and standard permissions models, customers can easily lift-and-shift applications to the cloud that need this capability.
Backing up data using existing mechanisms, software, and semantics can create an isolated recovery scenario with little locational flexibility for recovery. Many businesses want to take advantage of the flexibility of storing database backups in the cloud either for temporary protection during updates or for development and test. Since cloud file storage solutions present a standard filesystem that can be easily mounted from database servers, they can be an ideal platform to create portable database backups using native application tools or enterprise backup applications.
Development environments can be challenged to share data in a safe and secure way as they are collaborating to develop their latest innovations. With the need to easily share code and other files in an organized way, utilizing shared cloud file storage provides an organized and secure repository that is easily accessible within their cloud development environments. Cloud file storage delivers a scalable and highly available solution ideal for collaborating.
Containers are ideal for building microservices because they’re quick to provision, easily portable, and provide process isolation. Containers that need access to the original data each time it starts may require a shared file system that it can connect to regardless of which instance its running on. Cloud file storage can provide persistent shared access to data that all containers in a cluster can use. Serverless computing enables you to be more agile while spending less time dealing with the security, scalability and availability of your applications. AWS Lambda lets you run large-scale, and mission-critical serverless applications. Cloud file storage can provide highly available and durable serverless storage for the sharing of data that needs to persist beyond and between executions of AWS Lambda functions and AWS Fargate tasks.
A good file storage solution in the cloud must deliver the right performance and capacity for today and scale seamlessly as business needs change.
The benefits of cloud file storage are clear, but it is important to also note that not all cloud file storage solutions are created equal. A wide range of solutions exist. For example, a file storage solution might consist of a single node file server on a compute instance using block storage as its underpinnings with no scalability and few redundancies to protect data. Others are do-it-yourself clustered solutions requiring time-consuming setup, management and maintenance. And, there are fully managed solutions, such as Amazon Elastic File System (Amazon EFS), Amazon FSx for NetApp ONTAP, Amazon FSx for OpenZFS, Amazon FSx for Windows File Server, and Amazon FSx for Lustre, requiring little to no setup and maintenance, performing to meet the needs of the most demanding application workloads.
Cloud file storage can be delivered in one of two ways: through fully managed solutions with minimal setup and little to no maintenance, or through do-it-yourself solutions with separate compute, storage, software, and licensing, which require staffed expertise to configure and maintain. Amazon EFS, Amazon FSx for NetApp ONTAP, Amazon FSx for OpenZFS, Amazon FSx for Windows File Server, and Amazon FSx for Lustre are examples of fully managed solutions providing simple, scalable file storage for AWS compute models, including AWS containers and serverless, and on-premises use cases.
There are three types of cloud storage: Object, File, and Block. Each offers their own unique advantages.
1. Object Storage - Applications developed in the cloud often take advantage of object storage's vast scalability and metadata characteristics. Object storage solutions like Amazon Simple Storage Service (Amazon S3) are ideal for building modern applications from scratch that require scale and flexibility, and can also be used to import existing data stores for analytics, backup, or archive.
2. File Storage - Many applications need to access shared files and require a file system. This type of storage is often supported with a Network Attached Storage (NAS) server. File storage solutions like Amazon Elastic File System (EFS), Amazon FSx for NetApp ONTAP, Amazon FSx for OpenZFS, Amazon FSx for Windows File Server, and Amazon FSx for Lustre are for use cases like content repositories, development environments, machine learning, data science, media stores, and user home directories. Amazon FSx for Lustre is ideal for high-performance computing.
3. Block Storage - Other enterprise applications like databases or ERP systems often require dedicated, low latency storage for each host. This is analogous to direct-attached storage (DAS) or a Storage Area Network (SAN). Block-based cloud storage solutions like Amazon Elastic Block Store (EBS) are provisioned with each virtual server and offer the ultra-low latency required for high performance workloads.
Although object storage solutions enable storage of files as objects, accessing with existing applications requires new code and the use of API’s and direct knowledge of naming semantics. File storage solutions that support existing file system semantics and permissions models have a distinct advantage in that they do not require new code to be written to integrate with applications that are easily configured to work with shared file storage.
Block storage can be used as the underlying storage component of a self-managed file storage solution. However, the one-to-one relationship required between the host and volume makes it difficult to have the scalability, availability, and affordability of a fully managed file storage solution, and requires additional budget and management resources to support. Using a fully managed cloud file storage solution removes complexities, reduces costs, and simplifies management.
There’s a vast amount of file-based data in the world and AWS provides fully managed file system services that help you easily address the diverse needs of your file-based applications and workloads.
Organizations require their mission critical business applications to be highly available, and a great many of these applications use shared file storage. Migrating these applications to the cloud provides the scalability, high availability and durability, security, and reduced costs, while increasing agility.
AWS offers file system services optimized for your applications and use cases.
- Amazon EFS provides a simple, serverless, set-and-forget elastic file system that lets you share file data without provisioning or managing storage for a broad range of Linux-based applications.
- Amazon FSx for NetApp ONTAP provides fully managed highly reliable, scalable, performant, fully managed shared storage for Linux, Windows, and MacOS workloads.
- Amazon FSx for OpenZFS provides fully managed shared file storage built on the OpenZFS file system, powered by the AWS Graviton family of processors, and accessible via the NFS protocol (v3, v4, v4.1, v4.2).
- Amazon FSx for Windows File Server provides a fully managed native Windows file system with features and performance optimized for Windows-based business applications.
- Amazon FSx for Lustre is for compute-intensive applications such as high-performance computing. Amazon FSx for Lustre allows you to easily process data with a file system that’s optimized for the performance and cost of short-lived, compute-intensive processing jobs, with input and output stored on Amazon S3.