Access the elastic storage and throughput of Amazon S3 through a file interface
Mountpoint for Amazon S3 is an open source file client that you can use to mount an S3 bucket on your compute instance and access it as a local file system. It automatically translates local file system API calls to REST API calls on S3 objects. Mountpoint for Amazon S3 is optimized for high-throughput performance. It builds on the AWS Common Runtime (CRT) library, which is purpose-built for high performance and low-resource usage to make efficient use of your compute resources.
Benefits
File API access
Mountpoint for Amazon S3 presents S3 objects as files in the local file system and translates local file system API calls to REST API calls on S3 objects. It supports sequential and random read operations and sequential write operations for creating new files.
High throughput
Applications that use Mountpoint for Amazon S3 benefit from integration with the AWS CRT library, which implements best practice performance design patterns for S3 clients.
Open source
Customer feedback and contributions drive the evolution of Mountpoint for Amazon S3. To provide feedback and contribute, visit GitHub.
High reliability
You can reliably scale up and down over thousands of instances, processing petabytes of data in Amazon S3 data lakes.
Optimized repeated data access
You can cache data in Amazon EC2 instance storage, instance memory, or an Amazon EBS volume to improve the cost and performance of your application. To get started, read the documentation.
Support for Kubernetes applications
With the Mountpoint for Amazon S3 Container Storage Interface (CSI) driver, your Kubernetes applications can access S3 objects through a file system interface without any changes to your application code. To get started, read the documentation.
When to use Mountpoint for Amazon S3
Mountpoint for Amazon S3 is ideal for workloads that read large datasets (terabytes to petabytes in size) and require the elasticity and high throughput of Amazon Simple Storage Service (Amazon S3). Common use cases include large-scale machine learning (ML) training, autonomous vehicle simulation, genomics analysis, and image rendering. While these workloads read large datasets over several compute instances, they write sequentially to a file from a single node. This means they do not need shared file system features such as locking.
Mountpoint for Amazon S3 is designed around the tenet that it exposes the native performance of Amazon S3 and doesn’t support file system operations that can’t be efficiently implemented against S3 object APIs. This means Mountpoint for Amazon S3 doesn't try to emulate shared file system features that have no close analog in S3 object APIs. For applications that require shared file system features such as file locking and POSIX permissions, you can use Amazon FSx for Lustre with a data repository association to your S3 bucket.
How it works
Customers
Continental develops pioneering technologies and services for sustainable, connected mobility of people and their goods. Founded in 1871, this technology company offers safe, efficient, intelligent, and affordable solutions for vehicles, machines, traffic, and transportation.
"Continental’s Advanced Driver Assistance Systems (ADAS) technologies enable a high level of driving comfort and crash avoidance with features like Adaptive Cruise Control and Emergency Brake Assist. The validation of ADAS development requires large sets of input data stored in Amazon S3 to be re-simulated in compute workloads running on Amazon EC2 instances. Mountpoint for Amazon S3 allows us to reduce the idle time spent waiting to copy data to EC2 instances for processing. With its high-throughput performance, Mountpoint for Amazon S3 widens our choice of EC2 instance types, significantly reducing our compute costs."
The An Binh Nguyen, Product Owner for Cloud-based Simulation Platform, Continental
Untold Studios is a BAFTA-, EMMY-, and GRAMMY-nominated studio, shaping culture through music, TV, and advertising. Untold Studios develops original programming, produces music and advertising content, and crafts world-class VFX, all made possible by next-generation technology.
"Being able to quickly iterate on infrastructure and take advantage of new technologies is why Untold Studios runs on AWS. Mountpoint for Amazon S3 reduces the cost of storage and simplifies architecture for our render workflows that require high-throughput access to read and write TBs of transient files. It gives our applications direct access to Amazon S3's elastic throughput and storage over a file interface, freeing up performance and capacity on our primary file system for artists and projects."
Sam Reid, Head of Technology, Untold Studios
Amagi is a global media technology SaaS leader, providing end-to-end cloud-managed live video, on-demand video, and monetization solutions for broadcast and streaming TV.
"At Amagi, we rely heavily on Amazon S3 to store large amounts of video, audio, graphics, and metadata. In the past, we tried various FUSE-based file systems with S3-based storage infrastructure, but they failed to meet our SLA requirements for broadcast-grade playout systems. We are thrilled with the sustained high read throughput we have achieved in long-term experiments with Mountpoint for Amazon S3. We prioritize cost-effective solutions for our customers while maintaining high SLA standards, and Mountpoint for Amazon S3 will enable us to do so across multiple products."
Arpit Malani, Engineering Manager, Platform & Streaming TV, Amagi
Druva delivers data protection and management across endpoints, data centers and cloud workloads eliminating dependence on complex infrastructure and time-consuming administration.
"Our 4,000 customers need to meet expanding global data protection, compliance, and security requirements. Amazon S3 Object Lambda is a market-changing innovation that brings compute resources closer to centralized data repositories, making it possible for our customers to extract more value from their data. As a platform built entirely on AWS, Druva prides itself on continuously enhancing the user experience with AWS’s latest features and technology. The combination of the Druva Cloud Platform, powered by AWS, and Amazon S3 Object Lambda will bring organizations unmatched integration opportunities as well as direct access and analysis to unlock the full potential of their backup data."
Stephen Manley, Chief Technology Officer, Druva
Mountpoint for Amazon S3 resources
AWS News Blog
Mountpoint for Amazon S3 – Generally Available and Ready for Production Workloads
Read the blog post »Get started with Mountpoint for Amazon S3
Get started by downloading Mountpoint for Amazon S3 and mounting an S3 bucket on your compute instance using simple commands. Then you can access the bucket as a local file system as applications read and write objects as files that are locally available on your compute instance. For detailed instructions, read the Mountpoint for Amazon S3 User Guide.
Learn about Mountpoint for S3 by reading the user guide.
Get started building with Amazon S3 in the AWS Management Console.