AWS Links – Tuesday, January 27, 2009
A whole lot of interesting AWS items have landed in my inbox in the 20 days since my last links post. Here’s what I have queued up:
Chris Richardson wrote to tell me that there are still a few seats available in his upcoming half-day class, Running Java and Grails applications on Amazon EC2.
The class will take place on February 17th in Oakland, California. Chris, the author of POJOs in Action, creator of the Cloud Tools, and proprietor of the Cloud Foundry (recently reviewed in eWeek), will cover Amazon-style cloud computing, deep-dive into AWS, talk about the use of Amazon EC2, provide an overview of the Cloud Tools, and then jump into developing and deploying on EC2.
Tom Lounibos, CEO of SOASTA, dropped me a note to tell me that they recently used the EC2-powered CloudTest sytem to simulate the effect of 500,000 concurrent users downloading songs from a major media site. In an entry on the SOASTA blog, Tom notes that
These tests generate a substantial amount of data, ranging from 100 GB up to 1 TB in certain cases. SOASTA’s testing tool also provides real-time analytic processing of this huge amount of data.
Prabhakar from Ylastic wrote in to tell me that they have added full support for EC2‘s new EU region. They provide access to all EC2 resources (AMIs, access keys, security groups, elastic IPs, EBS volumes, and EBS snapshots), along with with a set of filtering and searching toools. Support for mobile (iPhone and Android) versions is almost ready and will be rolled out in the near future. Ylastic also tracks and organizes all EC2 alerts.
A few months ago Yuvi let me know that he needed a Wiki running in the EC2 cloud for a new project. After getting a good understanding of his needs I pointed him at the Mindtouch Deki product (previously blogged here). Yuvi wrote about his experiences in selecting and setting up the site, noting that “Our dev and prod sites were up and running in days! Incredible!”
Last week, in conjunction with the inauguration day festivities, they launched Whorunsgov.com. On the site you can get an insight into the people behind the scenes in Washington DC. You can see how deals get made and how policy is shaped. Read more on the site’s blog.
While I was putting the finishing touches on this post, I saw a Tweet from the folks at Mindtouch. They put together an excellent press release with even more information about the project and the selection criteria.
Jamal from Kaavo wrote in to tell me about the recent release of their IMOD product. IMOD stands for Infrastructure on Demand. IMOD uses an application-centric view of lifecycle management, managing anything from a single server to a complex multi-server system using their dynamic template technology. They also support goodies such as AED256 encryption of EBS volumes, monitoring of CPU, I/O, and memory usage, alerts for application service levels, and automatic backup of EBS volumes.
On Wednesday, February 4th, AWS evangelist Jinesh Varia will talk about use of AWS within the biotech community. Co-sponsored by Cirrhus9, the event will take place at Pfizer’s lab in La Jolla, California. Attendance is free but you need to register. You’d better hurry, there are just 17 seats left.
Simone and Alexis will each speak for 20-25 minutes, and there will be time for some Q & A after that. The meeting will then adjourn to the Crown Clerkenwall Green and Alexis will buy the first couple of rounds! Once again, registration is a must.
The Amazon WS Tools provide full support for CloudFront, SimpleDB, S3, SQS, and EC2. Each tool provides one or more views into the corresponding Amazon web service. For example, you can enter SimpleDB queries in the Query Editor, and you can see the results in the SimpleDB View, as you can see here.
The tools can be downloaded here. Bob provided me with the following installation instructions:
- Go to the [Help -> Software Updates -> Find and Install] Menu Item.
- Create a New Remote Site. Call is “jsLex Update Site” and reference http://www.rockstarapps.com/update-beta.
- Keep clicking through the screens until it is installed.
Pete from Juice Analytics wrote to tell me about their new Concentrate application. Concentrate is a search analytics tool for SEO and paid search professionals. It allows them to make better decisions, target SEO efforts, understand paid search campaigns, and better understand customer needs. Concentrate discovers and visualizes interesting patterns in the search queries used to locate a site. The word tree shown at right is just one form of output.
On the technical side, Concentrate runs on EC2. Here’s what Pete told me:
The back end consists of an EC2 cluster running Concentrate’s text mining algorithms. It accepts requests using a REST API and returns data in JSON format. The back end uses Amazon SQS and S3, and was patterned after the model found in our Grep The Web example. Scaling is handled using a combination of iClassify, Puppet, and Capistrano. The deployment infrastructure was built by HJK Solutions of Seatle.
Andrew from Job Bounty Hunter wrote to tell me that they’d launched the site and that every last bit of it is running on top of AWS.
Employers and recruiters can use the site to publish job advertisements, along with a cash bounty on each job. Bloggers and web publishers can post the ads on their site using a variety of widgets. Job seekers apply for the jobs via the widgets. If they are placed in the job and stay there for 90 days, the blogger or web publisher collects the bounty! As you can see from their bounty chart, there’s currently $7,800 up for grabs just a few days after their launch. This is a really cool way to monetize web traffic while also providing a very interesting service for the site’s readers.
Because the widgets are embedded in other sites, Andrew has no idea when a traffic surge might hit. They must be able to scale to match the traffic of all of the sites that are displaying their widgets. Of course, this need for elastic capacity made it a perfect for for AWS!
In the post, he talks about how Glue is used connect people and things, recognizing books, music, movies, and other everyday topics in hundreds of web sites and connecting people around the topics. He then goes on to discuss the reasons why a relational database won’t solve his problems, including scale and automatic partitioning and replication of his data. From there he describes his use of 30 SimpleDB domains to hold information about People, and another 30 about Things. He uses the djb2 hash algorithm to spread the information out across the available domains, and also stores data redundantly to obviate the need for joins.
You can take a picture of the card with your cell phone web cam, or otehr device and upload it to Lasso. After some image enhancement on EC2, the image is transcribed by a Mechanical Turk worker and deposited in Salesforce, an email, or a spreadsheet. Read all about this in the recent Information Week story, How The Cloud Enables A New Set Of Personal Applications.
The SimpleDB Management Tools integrate directly into Visual Studio 2008 and above. The tools support direct addition, editing, and deletion of SimpleDB data and domains and can also run SimpleDB queries. The tools plug in to the O/R Mapper, with direct dragging of SimpleDB domains from the management tool into the mapper.
And that’s all that I have time for today. I hope that you’ve enjoyed this glimpse in to some of the cool stuff that our developer community (now 490,000 members strong) has been up to. Send me information about what you are doing with AWS and I’ll do my best to fit you in.