Posted On: Nov 30, 2021
Amazon FSx for Lustre, a service that provides cost-effective, high-performance, scalable file systems for compute workloads, is making it even easier to process data residing in Amazon S3 by enabling your S3 bucket’s contents to be updated automatically as data is updated in an FSx for Lustre file system.
Integrated with Amazon S3, FSx for Lustre enables you to easily process S3 datasets with a high-performance file system. Before today, when linked to an S3 bucket, an FSx for Lustre file system transparently presented S3 objects as files and updated its contents automatically on an ongoing basis as objects were added to, changed in, or deleted from your S3 bucket. With today’s launch, FSx for Lustre can also automatically update the contents of the linked S3 bucket as files are added to, changed in, or deleted from the file system. With this feature, FSx for Lustre now provides fast file access for S3 datasets that is designed to keep data synchronized between the file system and S3 in both directions. You can now concurrently process your data using file-based and object-based workflows and share results in near real time between these workflows.
The automatic export feature is available at no additional cost on Amazon FSx for Lustre file systems with Persistent 2 deployment type in US East (N. Virginia), US East (Ohio), US West (Oregon), Canada (Central), Europe (Frankfurt), Europe (Ireland), and Asia Pacific (Tokyo). You can configure your file system to automatically export updates to S3 using the AWS Management Console, the AWS CLI, and AWS SDKs. Learn more about using automatic export with FSx for Lustre file systems in our AWS News blog, and the Amazon FSx documentation.