Amazon FSx for Lustre Offers New Options and Faster Speeds for Working with S3 Data
Amazon FSx for Lustre, a high-performance file system for machine learning, high performance computing, analytics, and other data processing workloads, is making it even easier to process data residing in Amazon S3 with new options for writing results back to S3 and up to 2x faster data transfer speeds between FSx for Lustre and S3.
Data processing workloads commonly require data to be presented via a fast and scalable POSIX file system interface, and commonly have input data sets that are stored on Amazon S3. With FSx for Lustre, you can quickly and easily spin up a performant and scalable file system linked to your Amazon S3 data, transparently access S3 objects as files, write results back to S3, and then spin down the file system.
Previously, when writing results back to S3 from an FSx for Lustre file system, your changed or new files were written as new objects in a predefined location within your S3 bucket. Now, you have the option to overwrite the original objects in your Amazon S3 bucket or write the new or changed files to a prefix you specify. These options allow for additional flexibility to incorporate FSx for Lustre into your data processing workflows.
In addition, Amazon FSx for Lustre now provides up to 2x the data transfer speeds between your Amazon S3 bucket and your FSx for Lustre file system, allowing you to complete your processing workloads even faster.
These enhancements are available on all new files systems in all regions where FSx for Lustre is available. Learn more about the new options for writing results to S3 and learn more about FSx for Lustre performance.