Amazon Elastic File System (Amazon EFS) provides simple, scalable, elastic file storage for use with AWS Cloud services and on-premises resources. It is easy to use and offers a simple interface that allows you to create and configure file systems quickly and easily. Amazon EFS is built to elastically scale on demand without disrupting applications, growing and shrinking automatically as you add and remove files, so your applications have the storage they need, when they need it. It is designed to provide massively parallel shared access to thousands of Amazon EC2 instances, enabling your applications to achieve high levels of aggregate throughput and IOPS that scale as a file system grows, with consistent low latencies. As a regional service, Amazon EFS is designed for high availability and durability storing data redundantly across multiple Availability Zones.
With these capabilities, Amazon EFS is well suited to support a broad spectrum of use cases, including web serving and content management, enterprise applications, media and entertainment processing workflows, home directories, database backups, developer tools, container storage, and big data analytics workloads.
How it works
When mounted on Amazon EC2 instances, an Amazon EFS file system provides a standard file system interface and file system access semantics, allowing you to seamlessly integrate Amazon EFS with your existing applications and tools. Multiple Amazon EC2 instances can access an Amazon EFS file system at the same time, allowing Amazon EFS to provide a common data source for workloads and applications running on more than one Amazon EC2 instance.
You can mount your Amazon EFS file systems on your on-premises datacenter servers when connected to your Amazon VPC with AWS Direct Connect. You can mount your EFS file systems on on-premises servers to migrate data sets to EFS, enable cloud bursting scenarios, or backup your on-premises data to EFS.
Amazon EFS is a fully managed cloud file storage service that provides scalable, shared file storage in the AWS Cloud. Learn more about cloud file storage and the benefits compared to other cloud storage solutions.
Cloud storage options provide a simple and scalable way to store, access, and share data over the Internet. Here, we compare Amazon EFS to other AWS Cloud storage offerings, helping you understand when to choose Amazon EFS.
Amazon EFS has a simple web services interface that allows you to create and configure file systems quickly and easily. The service manages all the file storage infrastructure for you, avoiding the complexity of deploying, patching, and maintaining complex file system deployments.
SHARED FILE STORAGE
Multiple Amazon EC2 instances and on-premises servers can simultaneously access an Amazon EFS file system, so applications that scale beyond a single instance can access a file system. Amazon EC2 instances running in multiple Availability Zones (AZs) within the same region can access the file system, so that many users can access and share a common data source.
Amazon EFS provides a file system interface and file system access semantics (such as strong data consistency and file locking). Amazon EC2 instances mount Amazon EFS file systems via the NFSv4 protocol, using standard operating system mount commands. You can also mount Amazon EFS file systems on your on-premises datacenter servers via the NFSv4 protocol when connected to your Amazon VPC with AWS Direct Connect.
Amazon EFS automatically and instantly scales your file system storage capacity up or down as you add or remove files without disrupting your applications, giving you just the storage you need while also reducing time-consuming administration tasks.
Amazon EFS is designed to provide the throughput, IOPS, and low latency needed for a broad range of workloads. With Amazon EFS, throughput and IOPS scale as a file system grows, and file operations are delivered with consistent, low latencies.
Amazon EFS provides the capacity you need, when you need it, without having to provision storage in advance. You pay for what you use, with no minimum commitments or up-front fees. To see an example of the TCO benefits of Amazon EFS, see here.
HIGHLY AVAILABLE AND DURABLE
Amazon EFS is designed to be highly available and durable. Each Amazon EFS file system object (i.e. directory, file, and link) is redundantly stored across multiple Availability Zones.
Amazon EFS provides a simple and secure way to move data from existing on-premises or in-cloud file systems, with the ability to copy files and directories at speeds up to 5x faster than standard Linux copy tools.
Amazon EFS allows you to tightly control access to your file systems through POSIX permissions. Use Amazon Virtual Private Cloud (Amazon VPC) to manage network access. Use AWS Identity and Access Management (IAM), to control access to Amazon EFS APIs. Encrypt your data at rest and in transit for a comprehensive solution securing both stored data and data in flight.
Amazon EFS provides the scalability, elasticity, availability, and durability to be the file store for enterprise applications and for applications delivered as a service. Its standard file system interface and file system semantics make it easy to migrate enterprise applications to the AWS cloud or to build new ones.
MEDIA & ENTERTAINMENT WORKFLOWS
Media workflows like video editing, studio production, broadcast processing, sound design, and rendering often depend on shared storage to manipulate large files. Strong data consistency model with high throughput and shared file access can cut the time it takes to perform these jobs and consolidate multiple local file repositories into a single location for all users.
Amazon EFS provides the scale and performance required for big data applications that require high throughput to compute nodes coupled with read-after-write consistency and low-latency file operations. Many analytics workloads interact with data via a file interface, rely on file semantics such as file locks, and require the ability to write to portions of a file. Amazon EFS supports the needed file systems semantics and can scale capacity as well as performance.
Amazon EFS can provide storage for organizations that have many users that need to access and share common data sets. An administrator can use Amazon EFS to create a file system accessible to individuals across an organization and establish permissions for users and groups at the file or directory level.
CONTENT MANAGEMENT & WEB SERVING
Amazon EFS can be used as a durable, high throughput file system for content management systems and web serving applications that store and serve information for a range of applications like web sites, online publications, and archives. Since Amazon EFS adheres to the expected file system semantics, file naming conventions, and permissions that web developers are accustomed to, it can easily integrate with web applications and can be used for a range of applications like web sites, online publications, and archives.
SOFTWARE DEVELOPMENT TOOLS
Amazon EFS enables your organization to be more agile and responsive to customer needs. Provision, duplicate, scale, or archive your development, test, and production environments with a few clicks. With the ability to share code and other files in an organized way, shared cloud file storage with EFS provides an organized and secure repository that is easily accessible within their cloud development environments. Amazon EFS delivers a scalable and highly available solution that is ideal for testing and development workloads.
Amazon EFS presents a standard file system that can be easily mounted from database servers. They can be an ideal platform to create portable database backups using native application tools or enterprise backup applications. Many businesses want to take advantage of the flexibility of storing database backups in the cloud either for temporary protection during updates or for development and test.
Containers are ideal for building microservices because they’re quick to provision, easily portable, and provide process isolation. Containers that need access to the original data each time they start, require a shared file system that they can connect to regardless of which instance they’re running on. Amazon EFS provides persistent shared access to file data, which is ideal for container storage.
HERE uses Amazon EFS to store 1.2 million artifacts in more than 300 repositories, adding up to more than 20 TB. The system serves 750,000 downloads and 50,000 uploads each day from more than 1,000 users and continuous-integration systems. The elasticity of Amazon EFS has helped HERE reduce total cost of ownership compared to managing its own NFS servers and storage.
“Using Amazon EFS, we don’t have to spend time or money overprovisioning to meet growing demand,” says Suresh Prem, principal systems engineer at HERE. “We use exactly as much as we need, and we only pay for what we use.”