New – Resource-Oriented Bidding for EC2 Spot Instances

Earlier this year we introduced the EC2 Spot fleet API. As I noted in my earlier post, this API allows you to launch and manage an entire fleet of Spot instances with one request. You specify the fleet’s target capacity, a bid price per hour, and tell Spot what instance type(s) you would like to launch. Spot fleet will find the lowest price spare EC2 capacity available, and then work to maintain the requested target capacity.

Today we are making the Spot fleet API even more powerful with the addition of bidding weights. This new feature allows you to create and place bids that are better aligned with the instance types and Availability Zones that are the best fit for your application. Each call to RequestSpotFleet includes a single bid price (expressed on a per-instance basis). This was simple to use, but the simplicity disallowed certain desirable features. For example, there was no way to launch a fleet that had at least 488 GiB of memory spread across two or more R3 (Memory Optimized) instances or least 128 vCPUs spread across a combination of C3 and C4 (both Compute Optimized) instances.

New Resource-Oriented Bidding
Our new resource-oriented bidding model will allow you to make Spot fleet requests of this type. Think of each instance in a fleet as having a number of “capacity units” of some resource that affects how many instances you need to create a fleet of the proper size. In my examples above, the resources might be GiBs of RAM or vCPUs. They could also represent EBS-optimized bandwidth, compute power, networking performance, or another (perhaps more abstract) application-specific unit. You can now request a certain number of capacity units in your Spot fleet, and you can indicate how many such units each instance type contributes to the total.

As a result, you can now use resources of multiple instance types, possibly spread across multiple Availability Zones, without having to be concerned about the instance types that are actually provisioned. Each call to RequestSpotFleet already includes one or more launch specifications, each one requesting instances of a particular type, running a specified AMI. You can now include one or both of the following values in each launch specification:

WeightedCapacity – How many capacity units an instance of the specified type contributes to the fleet. This value is multiplied by the bid price per unit when Spot fleet makes bids for Spot instances on your behalf. You can use this, for example, to indicate that you are willing to pay twice as much for an r3.xlarge instance with 30.5 GiB of memory as for an r3.large instance with  15.25 GiB of memory.

SpotPrice – An override of the bid price per unit specified in the request. You can use this value to introduce a bias into the selection process, allowing you to express a preference (for or against) particular instance types, Availability Zones, or subnets.

Here’s a launch specification that would represent the memory-centric example that I introduced earlier (I have omitted the other values for clarity):

You would then specify a Target Capacity of 488 (representing the desired fleet capacity in GiB) in your call to RequestSpotFleet, along with a Bid Price that represents the price (per GiB-hour) that you are willing to pay for the capacity.In this example, you are indicating that you are willing to pay 16 times as much for an r3.8xlarge instance as for r3.large.

EC2 will use this information to build the fleet using the most economical combination of available instance types, looking for instances that have the lowest Spot price per unit. This could be as simple as one of the following, using a single instance type:

• 2 x r3.8xlarge
• 4 x r3.4xlarge
• 8 x r3.2xlarge
• 16 x r3.xlarge
• 32 x r3.large

Or something more complex and heterogeneous, such as:

• 1 x r3.8xlarge and 2 x r3.4xlarge
• 2 x r3.4xlarge and 8 x r3.xlarge
• 8 x r3.xlarge and 16 x r3.large

Over time, as prices change and instances are interrupted due to rising prices, replacement instances will be launched as needed. This example assumes that your application is able to sense the instance type (and the amount of memory available to it) and to adjust accordingly. Note that that the fleet might be overprovisioned by a maximum of one instance in order to meet your target capacity using the available resources. In my example above, this would happen if you requested a fleet capable of storing 512 GiB. It could also happen if you make a small request and the cheapest price (per unit) is available on a large instance.

About Those Units
The units are arbitrary, and need not map directly to a physical attribute of the instance. Suppose you did some benchmarking and measured the transaction rate (in TPS) for a number of different instance types over time. You could then request a fleet capable of processing the desired number of transactions per second, while knowing that EC2 will give you the of instance types that are the most economical at any given point in time. As I have pointed out in the past, the Spot mechanism sits at the intersection of technology and business, and gives you the power to build systems and to write code that improves the bottom-line economics of your business! There’s a lot of room to be creative and innovative (and to save up to 90% over On-Demand prices) here.

You can also use this mechanism to prioritize specific Availability Zones by specifying a higher WeightedCapacity value in the desired zone. In this case, your launch specification would include two or more entries for the same instance type, with distinct Availability Zones and weights.

Requests can be submitted using the AWS Command Line Interface (AWS CLI) or via calls to RequestSpotFleet.

Available Now
This new functionality is available now and you can start using it today in all public AWS regions where Spot is available.

— Jeff;

PS – For more information about Spot instances, take a look at two of my recent posts: Focusing on Spot Instances and Building Price-Aware Applications.