AWS Developer Blog

Uploading to Amazon S3 with HTTP POST using the AWS SDK for .NET

by Norm Johanson | on | in .NET | | Comments

Generally speaking, access to your Amazon S3 resources requires your AWS credentials, though there are situations where you would like to grant certain forms of limited access to other users. For example, to allow users temporary access to download a non-public object, you can generate a pre-signed URL.

Another common situation is where you want to give users the ability to upload multiple files over time to an S3 bucket, but you don’t want to make the bucket public. You might also want to set some limits on what type and/or size of files users can upload. For this case, S3 allows you to create an upload policy that describes what a third-party user is allowed to upload, sign that policy with your AWS credentials, then give the user the signed policy so that they can use it in combination with HTTP POST uploads to S3.

The AWS SDK for .NET comes with some utilities that make this easy.

Writing an Upload Policy

First, you need to create the upload policy, which is a JSON document that describes the limitations Amazon S3 will enforce on uploads. This policy is different from an Identity and Access Management policy.

Here is a sample upload policy that specifies

  • The S3 bucket must be the-s3-bucket-in-question
  • Object keys must begin with donny/uploads/
  • The S3 canned ACL must be private
  • Only text files can be uploaded
  • The POST must have an x-amz-meta-yourelement specified, but it can contain anything.
  • Uploaded files cannot be longer than a megabyte.
{"expiration": "2013-04-01T00:00:00Z",
  "conditions": [ 
    {"bucket": "the-s3-bucket-in-question"}, 
    ["starts-with", "$key", "donny/uploads/"],
    {"acl": "private"},
    ["eq", "$Content-Type", "text/plain"],
    ["starts-with", "x-amz-meta-yourelement", ""],
    ["content-length-range", 0, 1048576]

It’s a good idea to place as many limitations as you can on these policies. For example, make the expiration as short as reasonable, restrict separate users to separate key prefixes if using the same bucket, and constrain file sizes and types. For more information about policy construction, see the Amazon Simple Storage Service Developer Guide.


Signing a Policy

Once you have a policy, you can sign it with your credentials using the SDK.

using Amazon.S3.Util;
using Amazon.Runtime;

var myCredentials = new BasicAWSCredentials(ACCESS_KEY_ID, SECRET_ACCESS_KEY);
var signedPolicy = S3PostUploadSignedPolicy.GetSignedPolicy(policyString, myCredentials);

Ideally, the credentials used to sign the request would belong to an IAM user created for this purpose, and not your root account credentials. This allows you to further constrain access with IAM policies, and it also gives you an avenue to revoke the signed policy (by rotating the credentials of the IAM user).

In order to successfully sign POST upload policies, the IAM user permissions must allow the actions s3:PutObject and s3:PutObjectAcl.

Uploading an Object Using the Signed Policy

You can add this signed policy object to an S3PostUploadRequest.

var postRequest = new S3PostUploadRequest 
    Key = "donny/uploads/throwing_rocks.txt",
    Bucket = "the-s3-bucket-in-question",
    CannedACL = S3CannedACL.Private,
    InputStream = File.OpenRead("c:throwing_rocks.txt"),
    SignedPolicy = signedPolicy

postRequest.Metadata.Add("yourelement", myelement);

var response = AmazonS3Util.PostUpload(postRequest);

Keys added to the S3PostUploadRequest.Metadata dictionary will have the x-amz-meta- prefix added to them if it isn’t present. Also, you don’t always have to explicitly set the Content-Type if it can be inferred from the extension of the file or key.

Any errors returned by the service will result in an S3PostUploadException, which will contain an explanation of why the upload failed.


Exporting and Importing a Signed Policy

You can export the S3PostUploadSignedPolicy object to JSON or XML to be transferred to other users.

var policyJson = signedPolicy.ToJson();
var policyXml = signedPolicy.ToXml();

And the receiving user can re-create S3PostUploadSignedPolicy objects with serialized data.

var signedPolicy = S3PostUploadSignedPolicy.GetSignedPolicyFromJson(policyJson);
vat signedPolicy2 = S3PostUploadSignedPolicy.GetSignedPolicyFromXml(policyXML);

For more information about uploading objects to Amazon S3 with HTTP POST, including how to upload objects with a web browser, see the Amazon Simple Storage Service Developer Guide.