Posted On: Jul 23, 2020
Amazon FSx for Lustre, which makes it easy and cost effective to launch and run the world’s most popular high-performance file system in AWS, is making it even easier to process data residing in Amazon S3 by enabling your file system’s contents to be updated automatically as data is added to or changed in an S3 bucket.
FSx for Lustre integrates with Amazon S3, making it easy to process data sets with a high-performance Lustre file system. When linked to an S3 bucket, an FSx for Lustre file system transparently presents S3 objects as files and allows you to write changed data back to S3. Until today, when you created an FSx for Lustre file system linked to an S3 bucket, the FSx file system imported the list of files and directories from your S3 bucket once, during file system creation. Now, you can also configure your FSx file system to update its contents automatically on an ongoing basis as objects are added to or changed in your S3 bucket.
With this feature, AWS customers who periodically collect and process new data, such as autonomous vehicle researchers and financial analysts, can easily use Amazon FSx for Lustre to process the latest data when it is added to or changed in their S3 bucket.
The automatic import feature is available on new Amazon FSx for Lustre file systems in all regions where FSx is available. You can configure your file system to import S3 updates by using the AWS Management Console, the AWS CLI, and AWS SDKs. Learn more about automatically updating FSx for Lustre file systems.