How can I check the integrity of an object uploaded to Amazon S3?

2 minute read
1

I want to upload an object to an Amazon Simple Storage Service (Amazon S3) bucket. Additionally, I want to verify the integrity of the uploaded object. How can I do that?

Short description

Follow these steps to verify the integrity of uploaded objects using the Content-MD5 header:

Note: When you use the Content-MD5 header, Amazon S3 checks the object against the provided Content-MD5 value. If the values do not match, you receive an error.

1.    Calculate the Content-MD5 value of the object.

2.    Verify the integrity of the uploaded object by passing the Content-MD5 value as a request header during the object upload.

Resolution

Calculate the Content-MD5 value of the object

Windows OS

If you're using a Windows operating system, you can use the Get-FileHash cmdlet from Microsoft PowerShell Utility to calculate MD5 digest, like this:

Get-FileHash \path\to\file -algorithm MD5 | Format-List

Note: The Get-FileHash cmdlet is available with Microsoft PowerShell Utility version 4.0 and later.

Here's an example output:

Algorithm : MD5
Hash      : C9A5A6878D97B48CC965C1E41859F034
Path      : \path\to\file

Then, apply base64-encoding to the calculated MD5 digest to get the required Content-MD5 value:

$hashString ='C9A5A6878D97B48CC965C1E41859F034'
$hashByteArray = [byte[]] ($hashString -replace '..', '0x$&,' -split ',' -ne '')
$ContentMD5 = [System.Convert]::ToBase64String($hashByteArray)
Echo $ContentMD5
yaWmh42XtIzJZcHkGFnwNA==

In this example, the output of Echo $ContentMD5, ("yaWmh42XtIzJZcHkGFnwNA=="), is the required Content-MD5 value.

Linux OS

If you're using a Linux operating system, run the following OpenSSL command to obtain the Content-MD5 value of your file:

openssl md5 -binary PATH/TO/FILE | base64

Verify the integrity of the uploaded object

When you use PutObject to upload objects to Amazon S3, pass the Content-MD5 value as a request header. Amazon S3 checks the object against the provided Content-MD5 value. If the values do not match, you receive an error.

The Content-MD5 request header can also be used with the S3 UploadPart API.


Related information

Common request headers

S3 error responses

AWS OFFICIAL
AWS OFFICIALUpdated 2 years ago