TransferManager for Android

The AWS SDK for Android has a high-level utility—TransferManager—for managing file transfers to Amazon S3. TransferManager is designed to make transfers between your application to Amazon S3 easy and convenient. Meanwhile, the smart logic behind it can provide transfers with better throughput and performance.

Why TransferManager

TransferManager can have several advantages over low-level APIs for file transfers.

• Better performance: It makes extensive use of multipart uploads. When dealing with a large file, it attempts to use multiple threads to upload multiparts of the file at once, so that throughput can be improved significantly.
• More convenient: It hides the complexity for you. It has several convenient methods that save you from creating request objects before transferring files. Also, it allows you to download or upload a directory recursively.
• Easy migration: If you are already using low-level APIs, don’t worry, as you will find the migration very easy. You can pass request objects to TransferManager just like you pass them to low-level APIs. I will show this to you in an example later.

How to Use TransferManager

With TransferManager, it’s as simple as the following to upload a file to Amazon S3:

AWSCredentials credential = new BasicAWSCredentials("access key", "secret key");

// TransferManager manages its own thread pool, so
// please share it when possible.
TransferManager manager = new TransferManager(credential);

// Transfer a file to an S3 bucket.
Upload upload = manager.upload(bucket, key, file);

The upload method is non-blocking and returns immediately with an Upload object. Wait for the upload to complete and get the result of this upload by making the following blocking call.

UploadResult result = upload.waitForUploadResult();

If you are interested in tracking the transfer progress, track it inside a while loop till the transfer is done, like so:

while (!upload.isDone()) {
    long transferred = upload.getProgress().getBytesTransfered();
    // publish progress
    ...
    Thread.sleep(200);
}

Easy Migration

It might be a great hassle to use TransferManager if you are used to low-level APIs to transfer files. However, you may find the migration more simple and painless after you read the following example. Your code probably looks like this:

// Put the image data into S3.
try {
    ...
    // Content type is determined by file extension.
    PutObjectRequest por = new PutObjectRequest(
        Constants.getPictureBucket(), Constants.PICTURE_NAME,
        imageFile);
    // Send the request.
    s3Client.putObject(por);
} catch (Exception exception) {
    result.setErrorMessage(exception.getMessage());
}

TransferManager can take a PutObjectRequest object to start the transfer. Any listener set to the object is passed to TransferManager. Therefore, a one-line change is required after TransferManager is initialized:

TransferManager manager = new TransferManager(s3Client);
...
// Put the image data into S3.
try {
    ...
    // Content type is determined by file extension.
    PutObjectRequest por = new PutObjectRequest(
        Constants.getPictureBucket(), Constants.PICTURE_NAME,
        imageFile);
    // Send the request.
    manager.upload(por);
} catch (Exception exception) {
    result.setErrorMessage(exception.getMessage());
}

Configuration

Underneath TransferManager is an AmazonS3Client. You can configure its client options such as user agent string, max retry attempts, connection timeout, etc. Please refer to ClientConfiguration for a complete set of configurable options.

ClientConfiguration s3Config = new ClientConfiguration();
// Sets the maximum number of allowed open HTTP connections.
s3Config.setMaxConnections(5);
// Sets the amount of time to wait (in milliseconds) for data
// to be transferred over a connection.
s3Config.setSocketTimeout(30000);

AmazonS3Client s3Client = new AmazonS3Client(credential, s3Config);
TransferManager manager = new TransferManager(s3Client);

TransferManager itself has two configurable options: minimum part size for upload parts, and threshold in bytes for when to use multipart uploads. They are configurable through TransferManagerConfiguration.

TransferManagerConfiguration tmConfig = new TransferManagerConfiguration();
// Sets the minimum part size for upload parts.
tmConfig.setMinimumUploadPartSize(5 * 1024 * 1024);
// Sets the size threshold in bytes for when to use multipart uploads.
tmConfig.setMultipartUploadThreshold(5 * 1024 * 1024);

manager.setConfiguration(tmConfig);

Amazon S3 Multipart Upload requires the minimum part size to be at least 5 MB. Make sure both options are greater or equal to 5 MB when you configure TransferManager.

After you read this post, I hope you find TransferManager useful. Using TransferManager provides convenience and performance benefits. If you have further questions, please let me know.

If you like building mobile applications that use cloud services that our customers use on a daily basis, perhaps you would like to join the AWS Mobile SDK and Tools team. We are hiring Software Developers, Web Developers, and Product Managers.