AWS Developer Tools Blog

Introducing presigned URL downloads in the AWS SDK for Java 2.x

Until now, the AWS SDK for Java 2.x could generate presigned URLs through S3Presigner but could not consume one to download an object. Developers had to fall back to a low-level HTTP client, forgoing the SDK’s built-in retries, metrics, and checksum validation. Starting with version 2.48.0, you can download Amazon Simple Storage Service (Amazon S3) objects from a presigned URL directly through the SDK pipeline, on both S3AsyncClient and the Amazon S3 Transfer Manager.

With a presigned URL, you can grant temporary, scoped access to a single Amazon S3 object without sharing long-term AWS credentials. A backend application signs a URL and passes it to a client over a secure channel such as an HTTPS response. That client then downloads the object directly from Amazon S3, with no AWS credentials of its own. For example, a media application’s backend can authorize access and return the URL to the client. The client then downloads a large video file from Amazon S3.

In this post, I show you how to use the new download APIs in three ways: a single-request download, a multipart parallel download, and a Transfer Manager download with progress tracking.

Getting started

Add the s3 module for the async client. To use managed, high-throughput downloads, also add the s3-transfer-manager module:

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>s3</artifactId>
</dependency>
<!-- Optional: only needed for Transfer Manager downloads -->
<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>s3-transfer-manager</artifactId>
</dependency>

Prerequisite: a presigned URL

These examples assume you already have a presigned URL for a GetObject request, generated by a backend application that holds AWS credentials. To generate one, see Work with Amazon S3 pre-signed URLs in the AWS SDK for Java 2.x Developer Guide. The following examples refer to this URL as presignedUrl, a java.net.URL returned by presigner.presignGetObject(presignRequest).url(). These examples assume a Signature Version 4 (SigV4) presigned URL, which is the default for the Java S3Presigner and modern AWS SDKs. If you generate the URL elsewhere (for example, with botocore or the AWS CLI v1), prefer SigV4. SigV2 is deprecated, is only honored in Regions launched before 2014, and the checksum features described later require SigV4.

Download an object with a presigned URL

Choose one of the following three options to download the object. Option 1 handles most downloads: it uses a single request and needs no extra client configuration, which suits typical object sizes. Use multipart (Option 2 or 3) for large objects; by default, a multipart-enabled client switches to multipart for objects larger than 8 MB.

Option 1: Single-request download

The new capability is available through S3AsyncClient.presignedUrlExtension(), the entry point for the presigned URL operations. Unlike regular operations such as getObject, these don’t use the client’s credentials. The SDK sends the request to the presigned URL without re-signing it, and the URL carries its own authentication. To download an object directly to a file, pass the request and a destination path:

S3AsyncClient s3AsyncClient = S3AsyncClient.create();

PresignedUrlDownloadRequest request = PresignedUrlDownloadRequest.builder()
        .presignedUrl(presignedUrl)
        .build();

CompletableFuture<GetObjectResponse> future =
        s3AsyncClient.presignedUrlExtension()
                     .getObject(request, Paths.get("/tmp/my-object"));

GetObjectResponse response = future.join();

You can transform the response any way the SDK supports. For example, to read the object into memory as bytes:

CompletableFuture<ResponseBytes<GetObjectResponse>> bytesFuture =
        s3AsyncClient.presignedUrlExtension()
                     .getObject(request, AsyncResponseTransformer.toBytes());

To fetch only part of an object, set range on the request. To make the download conditional, set ifMatch to the object’s ETag; if the ETag doesn’t match, Amazon S3 returns a 412 Precondition Failed error. The following request fetches the first 1 KB:

PresignedUrlDownloadRequest rangeRequest = PresignedUrlDownloadRequest.builder()
        .presignedUrl(presignedUrl)
        .range("bytes=0-1023")   // first 1 KB
        .build();

Option 2: Multipart download

For large objects, turn on multipart on the S3AsyncClient. When the client is multipart-enabled and you don’t specify a range, presignedUrlExtension() splits the download into ranged parts and fetches them in parallel. This uses the same getObject call as a single-request download:

S3AsyncClient s3AsyncClient = S3AsyncClient.builder()
        .multipartEnabled(true)
        .build();

CompletableFuture<GetObjectResponse> future =
        s3AsyncClient.presignedUrlExtension()
                     .getObject(request, Paths.get("/tmp/large-object"));

Option 3: Transfer Manager download

Use the S3 Transfer Manager when you want transfer-progress tracking in addition to the multipart throughput from Option 2. Presigned URL downloads are supported on the standard (Netty-based) S3AsyncClient, including when multipart is enabled. Build the Transfer Manager on a multipart-enabled standard S3AsyncClient, and avoid S3TransferManager.create(), which uses the CRT-based client (not supported here) when it is present on the classpath:

// Build a multipart-enabled standard client (not the CRT-based client).
S3AsyncClient s3AsyncClient = S3AsyncClient.builder()
        .multipartEnabled(true)
        .build();

S3TransferManager transferManager = S3TransferManager.builder()
        .s3Client(s3AsyncClient)
        .build();

PresignedDownloadFileRequest downloadRequest = PresignedDownloadFileRequest.builder()
        .presignedUrlDownloadRequest(r -> r.presignedUrl(presignedUrl))
        .destination(Paths.get("/tmp/large-object"))          // download to a file
        .addTransferListener(LoggingTransferListener.create()) // report progress
        .build();

// Start the download to the file. Multipart parts are fetched in parallel.
PresignedFileDownload download =
        transferManager.downloadFileWithPresignedUrl(downloadRequest);

// Wait for the download to finish and get the response.
CompletedFileDownload completed = download.completionFuture().join();

Transfer-progress listeners work as they do for other transfers, so you can track progress while the download runs. For example, you can update a progress bar.

If you need a custom result type instead of a file, use downloadWithPresignedUrl(...) with your own AsyncResponseTransformer. The response type is inferred from the transformer you supply. For example, AsyncResponseTransformer.toBytes() returns the object as bytes in memory:

PresignedDownloadRequest<ResponseBytes<GetObjectResponse>> request =
        PresignedDownloadRequest.builder()
                .presignedUrlDownloadRequest(r -> r.presignedUrl(presignedUrl))
                .responseTransformer(AsyncResponseTransformer.toBytes()) // infers the result type
                .build();

Download<ResponseBytes<GetObjectResponse>> download =
        transferManager.downloadWithPresignedUrl(request);

ResponseBytes<GetObjectResponse> result = download.completionFuture().join().result();

Verify data integrity with checksums

Turn on checksum mode when you generate the presigned URL to have Amazon S3 return checksums that the SDK can validate. How validation then works depends on how you download the object: the SDK validates a checksum for single-request downloads (Option 1), and for multipart downloads (Options 2 and 3) it validates each part for which Amazon S3 returns a checksum. Set checksum mode on the GetObjectRequest before you presign:

GetObjectRequest getObjectRequest = GetObjectRequest.builder()
        .bucket("amzn-s3-demo-bucket")
        .key("my-key")
        .checksumMode(ChecksumMode.ENABLED)
        .build();
// Pass getObjectRequest to presigner.presignGetObject(...) to produce the presigned URL.

When the URL is signed with checksum mode enabled, the SDK automatically sends the required x-amz-checksum-mode header and validates any checksum Amazon S3 returns. For a full-object download, Amazon S3 returns a checksum for the whole object. For the ranged part requests used by multipart downloads, Amazon S3 returns a checksum only when a part’s range aligns with the object’s original upload part boundaries; a single-part object downloaded with multipart therefore receives no per-part checksum.

Note that a URL signed with checksum mode is not directly browser-executable, because the download must include the x-amz-checksum-mode header. The SDK adds this header for you automatically.

Limitations

  • The AWS CRT-based S3 client is not supported. Presigned URL downloads work on the standard (Netty-based) S3AsyncClient and the Transfer Manager built on it. Calling the extension on the CRT-based client throws UnsupportedOperationException.
  • Pause and resume are not supported. Unlike a Transfer Manager FileDownload, a PresignedFileDownload cannot be paused and resumed.

Conclusion

In this post, I introduced presigned URL downloads, a new capability in the AWS SDK for Java 2.x that you can use to consume a presigned URL through the SDK’s own pipeline. You get the SDK’s built-in retries, metrics, and checksum validation while keeping the credential-free access model that makes presigned URLs so useful.

To get started, upgrade to AWS SDK for Java 2.x version 2.48.0 or later and try S3AsyncClient.presignedUrlExtension() or S3TransferManager.downloadFileWithPresignedUrl(...).

I’d love your feedback. Share it or request features by opening an issue in the AWS SDK for Java 2.x repository on GitHub.

Jency Joseph

Jency Joseph

Jency Joseph is a Software Development Engineer on the AWS SDK for Java and Kotlin team. A recent Master's graduate from the University of Southern California, she is passionate about API design and developer experience, building libraries and tools that make it simpler to use AWS services.