Amazon Elastic File System (Amazon EFS) provides simple, scalable, elastic file storage for use with AWS Cloud services and on-premises resources. It scales elastically on demand without disrupting applications, growing and shrinking automatically as you add and remove files. Amazon EFS file systems are distributed across an unconstrained number of storage servers, enabling file systems to grow to petabyte-scale providing simultaneous access to your data from Amazon EC2 instances and on-premises servers. Its distributed design avoids the bottlenecks and constraints inherent to traditional file servers. Amazon EFS is a regional service storing data within and across multiple Availability Zones (AZs) for high availability and durability.
Amazon EFS supports a broad range of use cases from highly parallelized, scale-out workloads that require the highest possible throughput to single-threaded, latency-sensitive workloads. Use cases such as lift-and-shift enterprise applications, big data analytics, web serving and content management, application development and testing, media and entertainment workflows, database backups, and container storage.
Elastic and Scalable
With Amazon EFS, storage capacity is elastic, growing and shrinking automatically as you add and remove files, so your applications have the storage they need – when they need it. Since capacity is elastic, there is no provisioning necessary and you will only be billed for what you use. Amazon EFS is designed to be highly scalable both in storage capacity and throughput performance. It can grow to petabyte scale and allows massively parallel access from Amazon EC2 instances to your data. With Amazon EFS, throughput and IOPS scale as a file system grows, and file operations are delivered with consistent, low latencies.
Amazon EFS provides high-performance, secure access for thousands of connections to a shared file system using a traditional file permissions model, file locking, and hierarchical directory structure via the NFSv4 protocol. It allows you to simultaneously share files between multiple Amazon EC2 instances across multiple AZs, regions, VPCs, and accounts as well as on-premises servers via AWS Direct Connect or AWS VPN. This is ideal for your business applications that need to share a common data source. For application workloads with many instances accessing the same set of files, Amazon EFS provides strong data consistency helping to ensure that any file read will reflect the last write of the file.
Amazon EFS is designed to provide the throughput, IOPS, and low latency needed for a broad range of workloads and offers two performance modes: General Purpose and Max I/O. General Purpose provides the lowest latency per file system operation and can achieve this for random or sequential IO patterns. Max I/O can scale to higher levels of aggregate throughput and operations per second and is ideal for highly parallelized applications that can scale out to thousands of Amazon EC2 instances.
Amazon EFS offers two throughput modes: Bursting and Provisioned. The throughput mode helps determine the overall throughput a file system can achieve. With Bursting Throughput, the throughput scales with the size of the file system, dynamically bursting as needed to support the spiky nature of many file-based workloads. Provisioned Throughput is designed to support applications that require higher dedicated throughput than the default Bursting mode and can be configured independently of the amount of data stored on the file system.
Storage Classes and Lifecycle Management
Amazon EFS offers a Standard and an Infrequent Access storage class. The EFS Infrequent Access (EFS IA) storage class is designed for files accessed less frequently. You can start using EFS IA by creating a new file system and enabling Lifecycle Management. With Lifecycle Management enabled, EFS automatically moves files not accessed for 30 days from the Standard storage class to the EFS IA storage class, reducing storage costs by up to 85%.
Highly available and durable
Amazon EFS is designed to be highly available and durable. All files and directories are redundantly stored within and across multiple Availability Zones in a region to prevent the loss of data from the failure of any single component. The distributed architecture of Amazon EFS provides data protection from an AZ outage, system and component failures, and network connection errors.
Security and Compliance
Amazon EFS allows you to tightly control access to your file systems through POSIX permissions and Amazon EFS security groups. POSIX permissions allow you to restrict access from hosts by user and group while security groups act as a firewall, and the rules you add define the traffic flow. Use Amazon Virtual Private Cloud (Amazon VPC) to manage network access and AWS Identity and Access Management (IAM) to control access to Amazon EFS APIs. Amazon EFS also meets many eligibility and compliance requirements to help you meet your regulatory needs. Click here for a list of compliance programs in scope for Amazon EFS.
Amazon EFS offers encryption for data at rest and in transit providing a comprehensive encryption solution to secure both your stored data and data in flight. Data at rest is transparently encrypted using encryption keys managed by the AWS Key Management Service (KMS), eliminating the need to build and maintain a key management infrastructure. Encryption of data in transit uses industry-standard Transport Layer Security (TLS) to secure network traffic without having to modify your applications. Refer to the user documentation on Encryption for more information about encrypting file system data.
Data transfer and backup
Moving your Amazon EFS file data can be managed simply with AWS DataSync.
AWS DataSync is a managed data transfer service that makes it faster and simpler to move data between on-premises storage and Amazon EFS. You can use DataSync to transfer active datasets over AWS Direct Connect or the internet at speeds up to 10 times faster than open-source tools. It does this without the need for you to modify your applications or write to an API. You can use the service for one-time data migrations, ongoing workflows with periodic synchronization, and replication for data protection and recovery. DataSync automatically handles many of the tasks that can slow down migrations or burden your IT operations, including infrastructure management, encryption, data validation, and data transfer orchestration.
AWS Backup is a fully managed backup service that makes it easy to centrally manage and automate backups of your Amazon EFS file systems removing the need for costly, custom solutions and manual processes. AWS Backup goes beyond backing up Amazon EFS and centralizes the back up of data across other AWS services in the cloud as well as on premises. As applications move to the cloud, their data can become distributed across multiple services, making it difficult to manage and consolidate backup activity without creating custom scripts and manual processes. Using AWS Backup, you can centrally configure and audit AWS resources, automate backup scheduling, set retention policies, and monitor backup activity.