Category: Amazon EC2

Amazon EC2 Helps Justin.TV Take on E3

by Jeff Barr | on | in Amazon EC2 |

Yesterday morning we received an email from the folks behind Justin.TV. They were preparing for a big splash at the E3 show and wanted to make sure that we were in a position to handle the increased load. We assured them that we were.

Justin and his posse visited E3 and did a live broadcast (available in archive form here). Everything worked really well.

After the event, they shared some impressive statistics with us. At peak they served up 3,500 simultaneous video streams representing over 1 gigabit per second of network bandwidth. Within minutes of noticing the heavy load, they were able to increase their streaming capacity by over 400%, and they paid only for the server capacity that they needed. Once the load subsided they killed off the extra capacity.

That’s how it works when you are Web-Scale!

— Jeff;

Renkoo’s Booze Mail: EC2, and the Pan-Galactic Gargle Blaster

by Jeff Barr | on | in Amazon EC2 |

Booze_mail My friends Adam Rifkin and Joyce Park are the co-founders of Renkoo. Renkoo simplifies the process of getting together with a bunch of friends for an event or simply for a tasty beverage or two.

Last month Adam told me that they were planning to launch a Facebook application and that they wanted to host it on Amazon EC2 in order to accommodate what they hoped would be a rapidly growing user base. I got him signed up for the beta and their application (Booze Mail) launched just a few days ago. Using this application you can send a virtual tasty beverage — anything from a simple glass of water to a throat-scorching Pan-Galactic Gargle Blaster — to anyone on your Facebook friends list.

As Adam explains, “With the pressure of rejection removed from the situation, a sender can make offers to deepen friendships without trepidation.” This is where things get even more interesting. The recipient of the virtual drink can redeem it (by printing out an “IOU” style coupon) or by using Renkoo’s scheduling facility to organize an actual, face-to-face meeting.

People are sending around virtual drinks at an ever-increasing rate and they’ve been using EC2 to good advantage. Looking at the Appaholic traffic graph I see that the user base has grown from less than 100 at the end of June to over 13,500 as of this morning. I don’t know how many EC2 instances they are using, but I do know (because they told me) that they aren’t too nervous about growing. As Adam says:

“Thank goodness for EC2. Otherwise this would be making me very very nervous because we have no hardware, rackspace, or bandwidth that can handle where I think we’re going.”

Instead, they are able to focus on the features and functionality of their application.

Congratulations to Adam and to Joyce on the launch!

— Jeff;

PS – Feel free to add me to your friends list if we’ve met….

Additional EC2 Beta Spots Now Open

by Jeff Barr | on | in Amazon EC2 |

Bull_fighter_charge As of this morning, we have cleared the Amazon EC2 waiting list!

Everyone on the list with a valid payment method in their account and acccess to Amazon S3 has been enabled for EC2.

We are continuing to add capacity and can now accomodate some additional beta signups. Simply create your AWS account, sign up for S3, and then go to to sign up for EC2.

— Jeff;

EC2 Best Practices

by Jeff Barr | on | in Amazon EC2 |

The Space Program Blog has some good EC2 Best Practices

These include: automate everything, scale early, and keep your database as small as possible. Refer to the post for more details, and please feel free to add your own suggestions there as comments.

— Jeff;

Amazon EC2 For Scientific Processing

by Jeff Barr | on | in Amazon EC2 |

Bioinformatics_for_dummies Mike Cariaso was kind enough to set up the Meetup in Bethesda for my upcoming trip to Washington, DC. Mike has done some pretty cool work with with Amazon EC2, setting up the mpiBLAST tool to run on EC2.

MPI, short for Message Passing Interface, is a standard for coordinating processing on supercomputer grids. MPIPCH2 is a popular implementation of MPI.

BLAST is the primary bioinformatics tool used to query genome sequences against an established database, or to match one sequence against another. The primary BLAST tool is run as an online service by the National Institute of Health.

Running BLAST over MPI lets BLAST run on a processing grid; this variant is called mpiBLAST.

Mike’s work builds on that of Peter Skomorch, who did the work needed to get MPIPCH2 running on Amazon EC2. Peter documented his work in a very informative set of blog posts:

That last post doesn’t actually reference EC2, but it is entertaining nonetheless. Part 2 ends with a parallel fractal calculation running on 5 EC2 instances!

By the way, I’m very interested in hearing about more academic and scientific uses of EC2. Please feel free to post a comment.

— Jeff;

Goplan goes AWS

by Jeff Barr | on | in Amazon EC2 |

Goplan Goplan – An online collaboration and project management tool – is a full fledged Ruby on Rails Application that is based on Amazon EC2 for hosting and Amazon S3 for storage. It took me less than 1 minute to get started with the app. No ugly big forms to fill out nor answer any difficult security questions like “What is your great grandfather’s middle name?” The best part is signup form has 4 simple fields to fill.

Felix interviewed Goplan’s developer. This blog post certainly shares some interesting insights into and Amazon EC2/S3.

He notes:

Tiago: The switch to EC2 came when we needed to scale the application. Not only was it cheaper to run it on EC2, it was also easier to scale during the launch. The ability to launch more instances on demand was very useful when we were featured on TechCrunch, digg, etc. Using S3 was a natural consequence of our move to EC2.

Most interesting part of the blogpost was the knowledge and experience the developer shares. Things like which S3 ruby library was used, WAL approach to data loss within PostGreSQL database, Munin to monitor the DB instances in case of failover and the best part on how a database can scale on EC2:

several ways: by changing that behaviour for read-only queries (it wouldnt even be a days worth of work), by using a middleware solution (such as SQL Relay) or by addressing that at the database level (using Master-Master replication). MySQL has internal replication solutions, PostgreSQL has PGCluster (I have some experience using it) and there is always the option of using more complex (and expensive) solutions such as Oracle databases.

This shows how smart programmers are finding cool and nifty ways to architect applications and move on to solve more interesting problems.

— Jinesh

update: Felix is the author of blog/interview

WeoCeo Video

by Jeff Barr | on | in Amazon EC2 |

Weogeoceo There’s a new WeoCEO video available. After watching this video you will be able to understand how the WeoCeo product dynamically starts, monitors, and terminates Amazon EC2 instances in response to actual machine load while maintaining a constant, externally visible domain name.

You’ll need to be familiar with the Unix command line in order to appreciate this video! Follow closely (there’s no audio track) for best results. If you don’t know too much about the command line, rest assured that the cpuburn command doesn’t actually start fires; it simply keeps the processor 100% busy! When the WeoCEO detects that the existing set of EC2 instances is too busy (as measured by the load average) it spawns additional instances (up to a predetermined maximum) to alleviate the load.

In just a few minutes you will see how WeoCEO is able to offer “spike insurance”, automatic load balancing, and stable IP addresses.

— Jeff;

Enomalism Amazon EC2 Migration Module

by Jeff Barr | on | in Amazon EC2 |

Enomalism The Enomalism Amazon EC2 Migration Module supports bi-directional migration of virtual machine images to and from Amazon EC2. In other words, you can easily and transparently move running images back and forth between your local Xen-based computing environment and Amazon EC2.

Using this tool you can do all sorts of interesting things including creation of hot spares, backup snapshots, load balancing, and access to additional compute capacity to handle sudden, unanticipated bursts of traffic. The product also converts between EC2 AMI’s, VMWare images, and the Enomalism Xen format using command-line tools. There’s support for VMcasting, and plans are afoot to support and run Windows, Solaris, and BSD Unix in the future.

Available in standard ($295, 5 EC2 instances), Professional ($395, 15 EC2 instances), and Enterprise ($1495, unlimited EC2 instances).

Check it out!

— Jeff;

Fastest way to Get Started with EC2 – EC2UI

by Jeff Barr | on | in Amazon EC2 |

In this age of speed where Electric Cars go 0 to 60 in 4 seconds, I found the fastest way to get started with Amazon EC2 :

…. With the new Firefox Plugin : EC2UI

Firefox lovers will love it. It takes less than 4 seconds to download and install (unless you are running at stone-age dialup speeds). Yes! its a Firefox plugin for Amazon EC2. Created by one of our ingenious Amazonian developers and the fastest way for anybody who would want to get started with Amazon EC2.

Point your Firefox browser here, Install the Plugin and restart Firefox, Key in your AWS credentials and Start spawning Amazon EC2 Instances.

Ec2ui_3 The plugin currently allows you to Manage AMIs, Manage your Instances, Keypairs, Security Groups and Permissions. View all the available public and private AMIs, launch your AMIs, terminate your instances – all within the powerful Firefox environment.

— Jin

Mounting Amazon S3 as a File System in Amazon EC2

by Jeff Barr | on | in Amazon EC2 |

We have a new article in our Developer Connection!

In Mounting Amazon S3 as a File System in Amazon EC2, AWS developer Bill Donahue shows how to use the S3InfiniDisk product to mount Amazon S3 as a file system in Amazon EC2. Using S3InfiniDisk, data written to a mounted Linux file system is written to S3, giving EC2 a permanent storage system.

The article covers starting an instance, installing the Java Development Kit, installing and configuring S3InfiniDisk, and more.

If you would like to write an article for the Developer Connection, take a look at our Co-Marketing page.

— Jeff;