AWS Developer Tools Blog

Announcing the developer preview of Transfer Manager for Amazon S3 in the AWS SDK for JavaScript v3

On October 15, 2026, we launched the developer preview of Transfer Manager (@aws-sdk/lib-transfer-manager). It gives JavaScript developers a simpler way to move large datasets to and from Amazon Simple Storage Service (Amazon S3) at high throughput.

Transfer Manager provides a set of high-level operations for high-throughput uploads and downloads. It coordinates multipart requests, concurrency, checksums, and data movement that applications would otherwise manage themselves.

In both Node.js and browsers, Transfer Manager supports object uploads and streamed object downloads with configurable concurrency, cancellation, and progress events. In Node.js, it also provides downloadToFile, uploadDirectory, and downloadDirectory operations. It uses worker threads to parallelize file and in-memory transfers so they can make better use of available compute, network, and storage resources.

Important: This package is for early access and feedback. We do not recommend it for production environments. For details about the developer preview phase, see the AWS SDKs and Tools maintenance policy.

Simplify and accelerate large Amazon S3 transfers

Transfer Manager runs multipart work concurrently to increase throughput, so you don’t have to schedule it yourself. This API provides five transfer operations:

  • upload – Uploads a file, buffer, or stream.
  • download – Downloads an object as a stream.
  • downloadToFile – Downloads an object directly to a local file.
  • uploadDirectory – Uploads a local directory.
  • downloadDirectory – Downloads an Amazon S3 prefix to a local directory.

Key features

Transfer Manager includes the following key features. Each feature lists the platforms it applies to:

  • Worker-thread parallelism (Node.js). Transfer Manager uses a configurable number of worker threads to parallelize per-part work for file and in-memory transfers, which reduces main-thread contention. Stream-based uploads do not yet use worker threads.
  • Connection distribution (Node.js). On the worker path, Transfer Manager distributes connections across resolved Amazon S3 IP addresses to avoid concentrating traffic on one Amazon S3 front-end IP address.
  • Direct disk writes for large downloads (Node.js). Transfer Manager writes parts concurrently to non-overlapping offsets in a preallocated temporary file, then moves the file into place after the download completes. Concurrent offset writes improve file-write throughput, and the final move into place prevents partial files at the destination. For sustained large downloads, the opt-in useODirect: true setting bypasses the OS page cache to reduce write-path stalls. We plan to make this setting the default in supported environments before general availability (GA).
  • Transfer controls and progress (Node.js and browsers). Transfer Manager supports configurable multipart thresholds, part size, and upload and download concurrency. It also allows cancellation with AbortSignal and reports initiation, progress, completion and failure events. Progress events report bytes transferred for object transfers, bytes and files transferred for directory transfers.

Throughput benchmark highlights

To evaluate Transfer Manager, we measured download throughput against the instance’s network and storage capacity and compared upload throughput with @aws-sdk/lib-storage. All tests ran on one Amazon Elastic Compute Cloud (Amazon EC2) m8ib.48xlarge instance with the following configuration:

  • 192 vCPUs and 768 GiB of RAM
  • 200 Gbps of network bandwidth
  • Nine Amazon Elastic Block Store (Amazon EBS) gp3 volumes configured as RAID 0, with approximately 144 Gbps of aggregate disk bandwidth

Download results

The JavaScript SDK had no comparable high-level download utility, so we report download performance as absolute throughput and as a percentage of the test system’s capacity rather than as a speedup comparison. For the 5 × 1 TiB download workload:

  • The file test with useODirect: true averaged 128.69 Gbps, about 89% of the test system’s approximately 144 Gbps aggregate disk bandwidth.
  • The RAM test, which drained response streams without writing to disk, averaged 193.83 Gbps, about 97% of the instance’s 200 Gbps network bandwidth.

The benchmark configuration includes workload-specific tuning. The file test opts into useODirect: true. For sustained, large-transfer file-I/O workloads, the benchmark also sets UV_THREADPOOL_SIZE to 128, following the guidance cited in the developer preview announcement.

These settings address OS page-cache pressure and libuv file-I/O limits, respectively. Tune them for your environment rather than treating them as universal defaults.

Upload results

Depending on the workload, Transfer Manager delivered 8 to 40 times higher upload throughput than @aws-sdk/lib-storage across uploads measured with both packages. These measurements are not performance guarantees. Results depend on the following factors:

  • Object size and count
  • Transfer settings, such as part size
  • Network capacity and disk performance
  • Transfer duration, particularly for shorter workloads that spend more of their run on multipart setup and connection ramp-up before reaching sustained throughput

You can review the full benchmark methods and results in the developer preview announcement on GitHub.

Get started and share feedback

With Transfer Manager, JavaScript developers have a simpler way to move large datasets to and from Amazon S3 at high throughput. To try the developer preview, follow these steps:

  1. Install the package from npm: npm install @aws-sdk/lib-transfer-manager
  2. Review the README for configuration details and examples
  3. Try the preview with non-production workloads

We are working toward general availability. Planned work listed in the GitHub announcement includes the following:

  • Make range-based downloads the default, so that download parallelism does not depend on upload part sizes
  • Enable O_DIRECT by default where supported, with a buffered-write fallback
  • Extend worker-thread support to stream-based uploads

We look forward to hear from you. Share feedback, questions or general thoughts through the GitHub announcement or the repository issue tracker.

 

Abhinav Goyal

Abhinav Goyal

Abhinav Goyal is an engineering manager at AWS in SDKs and Tools team responsible for AWS JavaScript SDK, AWS Python SDK (boto3) and AWS PHP SDK. Prior to AWS, he has 20+ years of technology leadership experience building large scale distributed systems for a variety of banking applications. In his free time, Abhinav likes to read, play table tennis and go for long walks. He holds a bachelor's degree from Indian Institute of Technology, Delhi (India).