I'm trying to upload a large file using the Amazon S3 console. Why is the upload failing?

Last updated: 2021-01-26

I'm trying to upload a large file (1 GB or larger) to Amazon Simple Storage Service (Amazon S3) using the console. However, the upload persistently fails and I'm getting timeout errors. How do I resolve this?

Resolution

For large files, Amazon S3 might separate the file into multiple uploads to maximize the upload speed. The Amazon S3 console might time out during large uploads because of session timeouts. Instead of using the Amazon S3 console, try uploading the file using the AWS Command Line Interface (AWS CLI) or an AWS SDK.

Note: If you use the Amazon S3 console, the maximum file size for uploads is 160 GB. To upload a file that is larger than 160 GB, use the AWS CLI, AWS SDK, or Amazon S3 REST API.

AWS CLI

First, install and configure the AWS CLI. Be sure to configure the AWS CLI with the credentials of an AWS Identity and Access Management (IAM) user or role. The IAM user or role must have the correct permissions to access Amazon S3.

Important: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent AWS CLI version.

To upload a large file, run the cp command:

aws s3 cp cat.png s3://docexamplebucket

Note: The file must be in the same directory that you're running the command from.

When you run a high-level (aws s3) command such as aws s3 cp, Amazon S3 automatically performs a multipart upload for large objects. In a multipart upload, a large file is split into multiple parts and uploaded separately to Amazon S3. After all the parts are uploaded, Amazon S3 combines the parts into a single file. A multipart upload can result in faster uploads and lower chances of failure with large files.

For more information on multipart uploads, see How do I use the AWS CLI to perform a multipart upload of a file to Amazon S3?

AWS SDK

For a programmable approach to uploading large files, consider using an AWS SDK, such as the AWS SDK for Java. For an example operation, see Upload an object using the AWS SDK for Java.

Note: For a full list of AWS SDKs and programming toolkits for developing and managing applications, see Tools to build on AWS.


Did this article help?


Do you need billing or technical support?