Fetch Object Data and Metadata from Amazon S3 (in a Single Call)

I came across an excellent question earlier this week on our support forums. The question was essentially, “How can I fetch object data and metadata from Amazon S3 in a single call?”

This is fair question, also one I did not have a good answer to. Amazon S3 returns both object data and metadata in a single GET object response, while AWS::S3::S3Object#readdoes not. But why?

How It Used to Be

Here is an example of how to get data from an object in S3.

obj = s3.buckets['my-bucket'].objects['key']
data = obj.read

Notice the #read method is returning the object data. This leaves no good place to return the metadata (returning multiple values from a function in ruby is generally frowned upon). In this case, the aws-sdk gem was getting the data and metadata from S3, but it was discarding the metadata.

The Best of Both Worlds

Last year we added support for streaming reads to AWS::S3::S3Object#read. If you pass a block to #read, then the data is yielded in chunks to the block.

File.open('filename', 'wb') do |file|
  obj.read do |chunk|
    file.write(chunk)
  end
end

Perfect! Since the #read method is yielding data in chunks, its return value becomes unused. This allowed me to patch the #read method to return the object metadata instead of nil.

resp = obj.read do |chunk|
  file.write(chunk)
end

resp #=> {:meta => {"foo" => "bar"}, :restore_in_progress => false, :content_type=>"text/plain", :etag=>""37b51d194a7513e45b56f6524f2d51f2"", :last_modified => 2013-02-06 12:54:39 -0800, :content_length => 94512, :data => nil}

You can checkout the new feature on our GitHub master branch now. This will be part of our next release.

If you see an issue with the AWS SDK for Ruby (aws-sdk gem), please, post an issue on our GitHub issue tracker!