Amazon FSx for Lustre Documentation
Amazon FSx for Lustre is a managed service that is designed to provide cost-effective, high-performance, scalable storage for compute workloads. FSx for Lustre is designed to offer shared storage with sub-ms latencies, up to terabytes per second of throughput, and millions of IOPS. FSx for Lustre file systems can also be linked to Amazon Simple Storage Service (Amazon S3) buckets, allowing you to access and process data concurrently from both a file system and from the Amazon S3 API.
Use with compute workloads
FSx for Lustre is designed to be compatible with many Linux-based AMIs, including Amazon Linux, Red Hat Enterprise Linux (RHEL), CentOS, Ubuntu, and SUSE Linux.
Import and export Amazon S3 data
Amazon FSx integrates natively with Amazon S3, enabling you to access your S3 data to run data processing workloads.
Using the AWS Console, you can create a file system that’s linked to one or more S3 buckets. FSx for Lustre is designed to present S3 objects as files and to allow you to write results back to S3. Amazon FSx for Lustre file systems are designed to update as objects are added to, changed in, or deleted from your linked S3 bucket. Amazon FSx for Lustre is also designed to track changes on your file system and keep your linked S3 bucket updated as files are added, modified, or deleted.
Use with compute services
Amazon FSx for Lustre is designed to be accessible from workloads running on Amazon EC2 instances or on on-premises computers/servers, as well as from containers running on Amazon Elastic Kubernetes Service (EKS).
Helps you optimize
Helps you reduce administrative burden and scale capacity
Using the Amazon FSx console, CLI, or API, you can create and scale a high-performance Lustre-based file system.
Multiple deployment types
FSx for Lustre offers a choice between scratch and persistent file systems for short-term and longer-term data processing. Scratch file systems can be a good choice for temporary storage and shorter-term processing of data. Scratch file systems are designed not to replicate data and not to persist if a file server fails. Persistent file systems can be a good choice for longer-term storage and workloads. Persistent file systems are designed to replicate data and replace file servers if they fail.
To help you support data protection of persistent file systems, Amazon FSx can also take incremental backups of your file system. Backups are stored in Amazon S3.
Multiple storage options
FSx for Lustre offers different storage classes. The SSD storage class delivers consistent low latencies for your dataset. The FSx Intelligent-Tiering storage class tiers data across its three access tiers and offers optional SSD read caching for frequently accessed data.
Manage consumption with storage quotas
You can use storage quotas to monitor and control user-and group-level storage consumption on your file systems, and to help ensure that users or groups are not able to consume excessive amounts of capacity.
Data compression
You can use data compression to help reduce storage consumption of both your file system storage and your file system backups. The data compression feature uses the LZ4 compression algorithm, which is designed to deliver high levels of compression without adversely impacting file system performance. Once data compression is enabled, FSx for Lustre is designed to automatically compress newly written files before they are written to disk and automatically uncompress them when they are read.
Security and compliance
Encryption
Amazon FSx for Lustre file systems are designed to be encrypted at-rest, and in-transit encryption is available in select regions.
Network isolation
You can access your Amazon FSx file system from endpoints in your Amazon VPC, which enables you to isolate your file system in your own virtual network. You can configure security group rules and control network access to your Amazon FSx file systems.
Resource-level permissions
Amazon FSx is integrated with AWS Identity and Access Management (IAM). This integration means that you can control the actions your AWS IAM users and groups can take to manage your file systems (such as creating and deleting file systems). You can also tag your Amazon FSx resources and control the actions that your IAM users and groups can take based on those tags.
Centralized backup and compliance management with AWS Backup
Amazon FSx is integrated with AWS Backup, enabling you to use fully managed, policy-based backup and restore capabilities for your Amazon FSx file systems.
Cross-region and cross-account backup
You can copy your Amazon FSx file system backups across AWS Regions, AWS accounts, or both.
Additional Information
For additional information about service controls, security features and functionalities, including, as applicable, information about storing, retrieving, modifying, restricting, and deleting data, please see https://docs.aws.amazon.com/index.html. This additional information does not form part of the Documentation for purposes of the AWS Customer Agreement available at http://aws.amazon.com/agreement, or other agreement between you and AWS governing your use of AWS’s services.