Notes From Berlin

Another post from the road In this case the road is a three-week trip to Ireland, Germany, Portugal, and Canada before returning to Seattle. This week Im in Berlin for Web 2.0 Expo and Interop; both are under one roof.

Todays announcement that were launching Amazon S3 in Europe has me really excited, given where Im posting this entry from. You can guess what the #1 request from developers has been on this trip: When will Amazon S3 host data in the EU?. Developers in the United States were also asking for S3 in Europe, because they were looking for lower latency. Of course the requests for hosting in the EU are as old as the service itselfand checking off a top developer request makes the Amazon S3 product team as happy as the customers who asked.

To support Europe we enhanced Amazon S3s API to include a new parameter for locality. The name of the parameter is LocationConstraint, and if you use it then buckets are created in the location you specify. Existing applications will not be affected, and will create buckets in the United States, because thats the default behavior. Pricing is slightly different in Europe; although not but much. Use the AWS Price calculator if you want to estimate what your costs will be.

ISVs Already Support Amazon S3 in Europe
Existing ISVs who built products on top of Amazon S3 have been effusive. For example, Saurabh Dani from Bucket Explorer said “We built Bucket Explorer as a simple and easy to use, yet powerful Interface for S3. The majority of our customers represent three segments: people looking to use Amazon S3 as an online collaboration tool, as a reliable store for off-site backups, or as a content delivery network. And our EU customers from all three segments expressed the need for an EU node of S3 to keep their data confined to the EU. With the integration of Bucket Explorer with S3/EU, we are very excited to be able to address to our customer needs.” The latest version of Bucket Explorer can be downloaded at www.bucketexplorer.com/be-download.html.

Bucket Explorer and a popular Firefox add-in known as S3 Organizer (available at www.rjonna.com) have enhanced their products to enable users to specify whether buckets are created in North America or Europe. Of course youll need to download the latest version in order to take advantage of this new feature.

Real-World S3 Examples
The launch of S3 in Europe is one entry in a long list of milestones. To me, real-world implementations are always the most interesting. One really interesting example that demonstrates the power of scale on demand is in a recent blog post by software developers who work for the New York Times.

The post is about their project to convert old newspaper articles to PDF format for access online (check this out by searching for Titanic iceberg on the front page of the Times, and choose the 1851-1980 archive in the drop-down box). From the blog: [I] determined that if I used only four machines, it could take some time to generate all 11 million article PDFs. But thanks to the swell people at Amazon, I got access to a few more machines and churned through all 11 million articles in just under 24 hours using 100 EC2 instances, and generated another 1.5TB of data to store in S3. (In fact, it worked so well that we ran it twice, since after we were done we noticed an error in the PDFs.)

Wow

Mike