Front-End Web & Mobile

Pause and Resume Amazon S3 Transfers Using the AWS Mobile SDK for Android

Amazon S3 Transfer Manager makes it easy for you to access Amazon S3 cloud storage from your mobile app. It provides an asynchronous mechanism to manage uploads and downloads between your app and Amazon S3. You can easily check the status of your transfers, add handlers to run code when a transfer completes, and more.

In Version 2 of the AWS Mobile SDK for Android, we introduced the ability to Pause and Resume transfers to/from Amazon S3. Via simple API calls, you can pause in-progress transfers, and resume paused transfers. In this blog post, we demonstrate how to use these new features.

Initialize S3 Transfer Manager

We recommend using Amazon Cognito as a credential provider to initialize the S3 Transfer Manager. You can use Amazon Cognito to securely access other AWS services from your mobile app without requiring your AWS account credentials. Amazon Cognito assigns your users a set of temporary, limited privilege credentials to access your AWS resources. For example, you can create a policy for an S3 bucket that allows a particular user to access only their own folder.

public static AWSCredentialsProvider getCredProvider(Context appContext) {   
   if(sCredProvider == null) {
      sCredProvider = new CognitoCachingCredentialsProvider(
         appContext,
         Constants.AWS_ACCOUNT_ID,
         Constants.COGNITO_POOL_ID,
         Constants.COGNITO_ROLE_UNAUTH,
         Regions.US_EAST_1);
      sCredProvider.refresh();    
    }
    return sCredProvider;
}

For details, see Amazon Cognito and How to Authenticate Users.

Initialize S3 Transfer Manager using the credentials provider created in the previous step.

TransferManager transferManager = new TransferManager(getCredProvider(this));

Download files from S3

Let’s first look at download. To download a file from S3:

Download download = transferManager.download(BUCKET_NAME, mKey, file);

To pause the in-progress transfer:

try {
    PersistableDownload persistableDownload = download.pause();
    //do something if we didn’t abort
} catch(PauseException e) {
    //do something if we aborted
}

download.pause() returns a PersistableDownload object that you can use to resume the download.

download = transferManager.resumeDownload(persistableDownload);

Upload files to S3

Similarly, to upload a file to S3:

Upload upload = transferManager.upload(BUCKET_NAME, fileName, file);

To resume paused transfer:

try {
    PersistableUpload persistableUpload = upload.pause();
} catch(PauseException e) {
    //do something if we aborted
}

upload.pause() returns a PersistableUpload object that you can use to resume the upload.

upload = transferManager.resumeUpload(persistableUpload);

Using tryPause with Uploads

There are two different ways to pause: pause() and tryPause(boolean). The latter method is necessary because it’s not always possible to pause a transfer. Transferring happens in chunks, so if a file is split into multiple chunks for transfer and we want to pause, we can just throw away our current chunk and start from there. However, if a file is transferred as only one chunk, and we follow the same method, we are just throwing away the whole file, thus making it equivalent to an abort. We can only pause transfers that happen in chunks, which means that we can’t pause transfers that are opened with an InputStream. By default, you can pause uploads for files with size greater than 16MB as that is the default minimum limit to upload files in chunks. You can change this default using TransferManagerConfiguration.setMultipartCopyThreshold(). Encryption can also prevent pausing.

You can use tryPause as follows:

// Upload a file to Amazon S3.
Upload upload = transferManager.upload(BUCKET_NAME, mKey, file);
 
// Initiate a pause with forceCancelTransfer as true. 
// This cancels the upload if the upload cannot be paused.
boolean forceCancel = true;
PauseResult<PersistableUpload> pauseResult = upload.tryPause(forceCancel);

The status of the pause operation can be retrieved using PauseResult.getPauseStatus() and can be one of the following.

  • SUCCESS – Upload is successfully paused.
  • CANCELLED – User requested to cancel the upload if the pause has no effect on the upload.
  • CANCELLED_BEFORE_START – User tried to pause the upload even before the start and cancel was requested.
  • NO_EFFECT – Pause operation has no effect on the upload. Upload continues to transfer data to Amazon S3.
  • NOT_STARTED – Pause operation has no effect on the upload because it has not yet started.

On a successful upload pause, PauseResult.getInfoToResume() returns an instance of PersistableUpload that can be used to resume the upload operation at a later time.

Resuming uploads in case App Crashes

In order to support resuming uploads/downloads during app crashes, PersistableUpload or PersistableDownload can be serialized to disk. Here is how you can achieve this:

public void saveTransferState(PersistableTransfer persistableTransfer, String fileName) {
  // Create a new file to store the information.
  File f = new File(fileName);
  if( !f.exists() ) f.createNewFile();
  FileOutputStream fos = new FileOutputStream(f);
  // Serialize the persistable transfer to the file.
  persistableTransfer.serialize(fos);
  fos.close();    
}

To ensure a transfer’s transfer state is saved automatically so that if it is stopped by an app crash it can be resumed, you can pass an instance of S3ProgressListener to TransferManager.upload or TransferManager.download that serializes the data to disk. The following example demonstrates how to achieve this:

PutObjectRequest putRequest = new PutObjectRequest(BUCKET_NAME, mKey,file);
// Upload a file to Amazon S3.
transferManager.upload(putRequest, new S3ProgressListener() {
  ExecutorService executor = Executors.newFixedThreadPool(1);
  @Override
  public void onPersistableTransfer(final PersistableTransfer persistableTransfer) {
        executor.submit(new Runnable() {
          @Override
          public void run() {
              saveTransferState(persistableTransfer, “resume-upload”);
          }
        });
    }
});

To de-serialize and resume upload:

String fileName = "resume-upload";
File f = new File (fileName);
if(f.exists()) {
  FileInputStream fis = new FileInputStream(f); 
  // Deserialize PersistableUpload information from disk. 
  PersistableUpload persistableUpload = PersistableTransfer.deserializeFrom(fis); 
  // Call resumeUpload with PersistableUpload. 
  transferManager.resumeUpload(persistableUpload); 
  fis.close();
}

Related Links