AWS Links – Tuesday, January 27, 2009

Chris Richardson wrote to tell me that there are still a few seats available in his upcoming half-day class, Running Java and Grails applications on Amazon EC2.

The class will take place on February 17th in Oakland, California. Chris, the author of POJOs in Action, creator of the Cloud Tools, and proprietor of the Cloud Foundry (recently reviewed in eWeek), will cover Amazon-style cloud computing, deep-dive into AWS, talk about the use of Amazon EC2, provide an overview of the Cloud Tools, and then jump into developing and deploying on EC2.

Tom Lounibos, CEO of SOASTA, dropped me a note to tell me that they recently used the EC2-powered CloudTest sytem to simulate the effect of 500,000 concurrent users downloading songs from a major media site. In an entry on the SOASTA blog, Tom notes that

These tests generate a substantial amount of data, ranging from 100 GB up to 1 TB in certain cases. SOASTA’s testing tool also provides real-time analytic processing of this huge amount of data.

Prabhakar from Ylastic wrote in to tell me that they have added full support for EC2‘s new EU region. They provide access to all EC2 resources (AMIs, access keys, security groups, elastic IPs, EBS volumes, and EBS snapshots), along with with a set of filtering and searching tools. Support for mobile (iPhone and Android) versions is almost ready and will be rolled out in the near future. Ylastic also tracks and organizes all EC2 alerts.

I met Yuvi Kochar, CTO of The Washington Post, at a DC-area CTO event a year or two ago. We’ve stayed in touch and I paid a visit to him and his team last year on one of my trips east.

A few months ago Yuvi let me know that he needed a Wiki running in the EC2 cloud for a new project. After getting a good understanding of his needs I pointed him at the Mindtouch Deki product (previously blogged here). Yuvi wrote about his experiences in selecting and setting up the site, noting that “Our dev and prod sites were up and running in days! Incredible!”

Last week, in conjunction with the inauguration day festivities, they launched Whorunsgov.com. On the site you can get an insight into the people behind the scenes in Washington DC. You can see how deals get made and how policy is shaped. Read more on the site’s blog.

While I was putting the finishing touches on this post, I saw a Tweet from the folks at Mindtouch. They put together an excellent press release with even more information about the project and the selection criteria.

Jamal from Kaavo wrote in to tell me about the recent release of their IMOD product. IMOD stands for Infrastructure on Demand. IMOD uses an application-centric view of lifecycle management, managing anything from a single server to a complex multi-server system using their dynamic template technology. They also support goodies such as AED256 encryption of EBS volumes, monitoring of CPU, I/O, and memory usage, alerts for application service levels, and automatic backup of EBS volumes.

Read more on the IMOD Fact Sheet or watch some of the following videos (each conveniently hosted on Amazon S3):

• • IMOD Demo

IMOD Template Overview

• IMOD Quick Start

On Wednesday, February 4th, AWS evangelist Jinesh Varia will talk about use of AWS within the biotech community. Co-sponsored by Cirrhus9, the event will take place at Pfizer’s lab in La Jolla, California.

Also (coincidentally) on February 4th, AWS Evangelist Simone Brunozzi and CohesiveFT co-founder Alexis Richardson will be hosting an AWS Meetup in London.

Simone and Alexis will each speak for 20-25 minutes, and there will be time for some Q & A after that. The meeting will then adjourn to the Crown Clerkenwall Green and Alexis will buy the first couple of rounds! Once again, registration is a must.

Bob from Rockstar Apps wrote to tell me about their new Amazon WS tools. The tools plug in to Eclipse and Aptana, where they are available as the “Amazon WebService Perspective.”

The Amazon WS Tools provide full support for CloudFront, SimpleDB, S3, SQS, and EC2. Each tool provides one or more views into the corresponding Amazon web service. For example, you can enter SimpleDB queries in the Query Editor, and you can see the results in the SimpleDB View.

The tools can be downloaded here. Bob provided me with the following installation instructions:

1. Go to the [Help -> Software Updates -> Find and Install] Menu Item.
2. Create a New Remote Site. Call is “jsLex Update Site” and reference http://www.rockstarapps.com/update-beta.
3. Keep clicking through the screens until it is installed.

Pete from Juice Analytics wrote to tell me about their new Concentrate application. Concentrate is a search analytics tool for SEO and paid search professionals. It allows them to make better decisions, target SEO efforts, understand paid search campaigns, and better understand customer needs. Concentrate discovers and visualizes interesting patterns in the search queries used to locate a site. The word tree shown at right is just one form of output.

Concentrate is available under a number of different pricing plans, ranging from Free all the way up to Max. You can also start out by looking at the online demo.

On the technical side, Concentrate runs on EC2. Here’s what Pete told me:

The front end is powered by the DJango framework and the jQuery JavaScript library, all load balanced atop a number of EC2 instances. They run MySQL and store their data on EBS volumes.

The back end consists of an EC2 cluster running Concentrate’s text mining algorithms. It accepts requests using a REST API and returns data in JSON format. The back end uses Amazon SQS and S3, and was patterned after the model found in our Grep The Web example. Scaling is handled using a combination of iClassify, Puppet, and Capistrano. The deployment infrastructure was built by HJK Solutions of Seattle.

Andrew from Job Bounty Hunter wrote to tell me that they’d launched the site and that every last bit of it is running on top of AWS.

Employers and recruiters can use the site to publish job advertisements, along with a cash bounty on each job. Bloggers and web publishers can post the ads on their site using a variety of widgets. Job seekers apply for the jobs via the widgets. If they are placed in the job and stay there for 90 days, the blogger or web publisher collects the bounty! As you can see from their bounty chart, there’s currently $7,800 up for grabs just a few days after their launch. This is a really cool way to monetize web traffic while also providing a very interesting service for the site’s readers.

Because the widgets are embedded in other sites, Andrew has no idea when a traffic surge might hit. They must be able to scale to match the traffic of all of the sites that are displaying their widgets. Of course, this need for elastic capacity made it a perfect for for AWS!

They decided to serve up all of the widgets as static HTML. They are created using PHP and then downloaded using Curl into static HTML. The static HTML is then deployed to CloudFront (via S3) and Nginx. The front end of the site was implemented using HTML, CSS, Adobe Flex, JavaScript, and (again) jQuery.

Job Bounty Hunter currently runs on three EC2 instances. The first runs the free version of IBM’s DB2 database. The second one runs Apache, Postfix, PHP, and some web service code. The third instance runs Nginx and serves up static content such as widgets, JavaScript, images, HTML, and SWF files.

Andrew also sent along a very nice architecture diagram as an image and as PDF.

Long-time AWS user Alex Iskold has written an illuminating post about his use of Amazon SimpleDB for his new Glue product.

In the post, he talks about how Glue is used connect people and things, recognizing books, music, movies, and other everyday topics in hundreds of web sites and connecting people around the topics. He then goes on to discuss the reasons why a relational database won’t solve his problems, including scale and automatic partitioning and replication of his data. From there he describes his use of 30 SimpleDB domains to hold information about People, and another 30 about Things. He uses the djb2 hash algorithm to spread the information out across the available domains, and also stores data redundantly to obviate the need for joins.

Lasso2GGO uses a number of different services including S3, EC2, and the Mechanical Turk. This service makes it easy to convert “analog” business cards into handy digital data.

You can take a picture of the card with your cell phone web cam, or other device and upload it to Lasso. After some image enhancement on EC2, the image is transcribed by a Mechanical Turk worker and deposited in Salesforce, an email, or a spreadsheet. Read all about this in the recent Information Week story, How The Cloud Enables A New Set Of Personal Applications.

Finally, John-David from New Zealand’s Mindscape wrote in to tell me about their new products: SimpleDB Management Tools for Visual Studio and LightSpeed – The Mindscape .Net O/R Mapper.

The SimpleDB Management Tools integrate directly into Visual Studio 2008 and above. The tools support direct addition, editing, and deletion of SimpleDB data and domains and can also run SimpleDB queries. The tools plug in to the O/R Mapper, with direct dragging of SimpleDB domains from the management tool into the mapper.