New on the Open Source blog: Using the FSx for Lustre CSI Driver with Amazon EKS

From time to time the AWS Storage Blog will point you to storage related blog posts from other AWS blog channels that we think you will find interesting or helpful for your storage use cases. We wanted to share a post from the Open Source Blog that highlights a walk-through of using Amazon FSx for Lustre Container Storage Interface (CSI) Driver with Amazon EKS.

The Amazon Elastic Container Service for Kubernetes (Amazon EKS) team has been busy building CSI drivers for all our storage solutions, including Amazon FSx for Lustre: a fully-managed file system which has integrations with S3 and is optimized for compute-intensive workloads such as high-performance computing (HPC) and machine learning. Because the AWS FSx CSI Driver is potentially useful to any Kubernetes user, we’ve donated it to Kubernetes SIG-AWS. This blog post focuses on deploying the AWS FSx CSI driver to an Amazon EKS cluster.

You can now dynamically provision Amazon FSx for Lustre filesystems for your high performance computing workloads, and have a container automatically connected into this filesystem. FSx for Lustre provides a high-performance file system optimized for fast processing of workloads such as machine learning, high performance computing (HPC), video processing, financial modeling, and electronic design automation (EDA). These workloads commonly require data to be presented via a fast and scalable file system interface, and typically have data sets stored on long-term data stores like Amazon S3.

You can also use dynamic provisioning to consume the same persistent volume claim from multiple pods from different nodes. If you need to start your containers with a dataset automatically available, for example loading a machine learning training data set, you can even connect your StorageClass to an existing Amazon S3 bucket by using Dynamic Provisioning with Data Repository. Other examples can be found in the FSx for Lustre CSI Driver documentation.

To learn more, see the full AWS Open Source Blog post.