EC2 Spot Pricing – Now Specific to Each Availability Zone

We have made an important change to the way pricing works for EC2 Spot Instances. We are replacing the original Region-based pricing information with more detailed information that is specific to each Availability Zone. This change ensures that both the supply (the amount of unused EC2 capacity) and the demand (the amount and number of bids for the capacity) reflect the status of a particular Availability Zone, enabling you to submit bids that are more likely to be fulfilled quickly and to use Spot Instances in more types of applications.

As you may know, Spot Instances allow you to bid for unused EC2 capacity, often allowing you to significantly reduce your Amazon EC2 bill. After you place a bid, your instances continue to run as long as the bid exceeds the current Spot Market price. You can also create persistent requests that will automatically be considered again for fulfillment if you are ever outbid.

Over the last year and a half, our customers have successfully leveraged Spot Instances to obtain compute cycles at substantial discounts for use cases like batch processing, scientific research, image processing and encoding, data and web crawling, testing, and financial analysis. Here are some examples:

Social analytics platform BackType (full case study) uses Spot Instances to handle large-scale Hadoop-based batch data processing (tens of terabytes of data representing over 100 billion records). They have been able to reduce their costs by up to 66% when compared to On-Demand instances. Their Spot strategy includes placing high bids to reduce the chance of being interrupted (they pay the current price regardless of their bid, so this does not increase their operating costs).

Monitoring and load testing company BrowserMob (full case study) uses a combination of Spot and On-Demand instances to meet their capacity needs. Their provisioning system forecasts capacity needs 5 minutes ahead of time and submits suitably priced bids for Spot Instances, resorting to On-Demand instances as needed based on pricing and availability.

Biotechnology drug design platform Numerate (full case study) has incorporated Amazon EC2 as a production computational cluster and Amazon S3 for cache storage. Numerate enjoys around 50% cost savings by using Amazon EC2 Spot Instances after spending just 5 days of engineering effort.

Image rendering tool Litmus (full case study) takes snapshots of an email in various email clients and consolidates the images for their customers. Litmus enjoys a 57% cost savings by using Amazon EC2 Spot Instances for their compute needs.

When Spot Instances were first launched, there was a Spot Price for each EC2 instance type and platform (Linux/Unix or Windows) in each Region:

This model worked well but we think we can do even better based on your feedback. We have made some improvements to the Spot Instance model to make it easier for you to implement a cost-effective bidding strategy.

1. The market price for each type of Spot Instance is now specific to an Availability Zone, not a Region.
2. We will now publish historical pricing for each Availability Zone. This change will provide fine-grained data that you can use to determine a suitable bid for capacity in a particular Availability Zone. Because this will make a great deal of additional data available to you, we have made an API change to allow you to paginate the results of the DescribeSpotPriceHistory function.
3. Spot requests that target a particular Availability Zone now have a greater chance of being fulfilled.

With these changes you now have the information needed to do additional fine tuning of your Spot requests. In particular, we believe that these changes will allow you to more easily use Spot Instances for applications that are sensitive to latency or that need to run in a particular Availability Zone in order to be co-located with other services or data. For example,  it will now be easier for AWS to run a set of Hadoop-based processed in the same Availability Zone, paving the way for you to use Spot Instances with Elastic MapReduce.

Its easy to get started. Simply go to the AWS Management Console and launch an instance like normal. In the Request Instance Wizard, click the Request Spot Instance radio button to set your bid to the maximum that you are willing to pay for an instance hour of the desired instance type. Here’s a screenshot of a persistent request for four Micro instances at a maximum price of $0.01 (one penny) per hour per instance:

I will look forward to hearing about the ways that you find to put Spot Instances to use in your application.

Read More:

• Werner Vogels: Spot Instances – Increased Control.
• EC2 Documentation: Introduction to Spot Instances.
• EC2 Spot Instances Home Page.

— Jeff;