AWS Case Study: BrowserMob (Now Neustar Web Performance)

To conduct its load testing services, BrowserMob uses Amazon Elastic Compute Cloud (Amazon EC2). The company is leveraging Amazon EC2 On-Demand and Spot Instances. BrowserMob will start by launching a baseline of On-Demand instances to ensure the capacity will run for the entire duration of the test. BrowserMob’s provisioning continuously looks ahead 5 minutes to determine their capacity needs. It will attempt to launch a Spot Instance first at a lower bid price. If the instance doesn’t launch within the first 5-7 minutes, BrowserMob will attempt to run On-Demand. If the provisioning process still can’t get the required instances using Amazon EC2 On-Demand, it bids on Spot Instances, which allow BrowserMob to access unused Amazon EC2 instances that have not been available with the On-Demand pricing.

The cost of the Amazon EC2 Spot Instances is based on a fluctuating Spot Price that is determined by the current supply and demand. BrowserMob is able to use the Spot Instances as long as its bid is higher than the Spot Price and the requested capacity is available. Patrick Lightbody, Founder of BrowserMob, explains that, “Spot Instances provide two advantages: First, we get a lower price when the capacity is available, and second, we get extra capacity when the On-Demand market is saturated. Sometimes bidding more than the On-Demand price is worthwhile in order to get the extra capacity.”

BrowserMob realizes that it can potentially be outbid for the Amazon EC2 Spot Instances it is using, in which case, those instances will stop running. The company prepares for that situation by backing up its data to Amazon Relational Database Service (Amazon RDS) every ten seconds while using Spot Instances. Additionally, BrowserMob has engineered their processes to automatically provision On-Demand instances in the case of failure.

By combining the capacity provided by Amazon EC2 On-Demand Instances and Spot Instances, BrowserMob is able to run approximately 1000 Amazon EC2 instances for less than one hour intervals twenty to thirty times every day. Additionally, BrowserMob is able to further expand its Amazon EC2 usage when necessary to meet the needs of its customers. For example, the company reports that it ran 5,000 Firefox browsers on 2,300 CPU cores within Amazon EC2 in order to load test an online gaming Website that was promoting the launch of a major movie.

BrowserMob’s Website monitoring services also use Amazon EC2, as well as Amazon Simple Queue Service (Amazon SQS). Together, the various Amazon Web Services employed by BrowserMob give the company predictable expenses and help it to avoid the complications of maintaining its own physical infrastructure. Lightbody says, “An interesting benefit of AWS is that our projected costs as a function of revenue and product usage have had a variance from the actual costs by less than one percent since we launched BrowserMob. That is a big deal for us!”

BrowserMob is currently exploring options to create additional cost savings through Amazon EC2 Spot Instances by allowing its customers the flexibility to determine their own service needs in relation to the current Spot Price. In addition, BrowserMob plans to include the AWS infrastructure in the development of new load testing and monitoring services related to cellular network latency and smartphone Web browsers.

Lightbody leaves no doubt about the value of AWS to BrowserMob: “AWS is crucial for us. BrowserMob could not have been started without it. Our approach to load testing depends on launching thousands of instances in short order, and AWS fits that model perfectly. So, in that sense, the only metric that shows the benefit of Amazon EC2 is ‘zero’—that is the revenue we would be making without it.”

To learn more, visit http://browsermob.com/ .

Added April 21, 2011