Amazon EMR announces instance fleets for Amazon EC2 Spot instances and Spot blocks
Amazon EMR now supports a variety of new Amazon EC2 Spot instance management features with instance fleets. Instead of specifying a specific Amazon EC2 instance type for an Amazon EMR instance group or Amazon EC2 availability zone for your Amazon EMR cluster, you can now provide a list of possible choices and allow Amazon EMR to automatically select an optimal combination. When creating clusters with instance fleets, Amazon EMR can now automatically provision Spot capacity across a variety of instance types, select optimal Amazon EC2 availability zones, and blend Spot and On-Demand capacity to minimize overall cost.
When creating an Amazon EMR cluster with instance fleets, you provide the target Spot and On-Demand Amazon EC2 capacity for your cluster and a list of possible instance types and related Spot bid prices to choose from. Next, Amazon EMR will automatically provision Spot and On-Demand capacity across your specified instance types to reach your target capacity. Additionally, Amazon EMR instance fleets support Spot blocks, or Spot instances for predefined durations, which allow you to execute without interruption for 1 to 6 hours.
You can create Amazon EMR clusters with instance fleets from the Amazon EMR console, AWS Command Line Interface (CLI), or the AWS SDK with the Amazon EMR API. Instance fleets are supported on Amazon EMR release 5.1.0 or later and 4.8.0 or later. When using instance fleets, you can specify a list of possible VPC subnets or Amazon EC2 Availability Zones where Amazon EMR can launch your cluster. Please visit the Amazon EMR documentation for more information about instance fleets.