Posted On: Jun 24, 2021
You can now specify up to 30 EC2 instance types for each of your master, core, and task node groups when using EMR Instance Fleets with allocation strategy. Previously, this was limited to 15 for tasks nodes, and only 5 for master and core nodes. This increase allows you to specify a broader range of instance sizes, generations, and families that your workloads can operate across to improve your access to both Spot and On-Demand capacity.
You can use the increased limits to increase access to Spot capacity and reduce the risk of Spot interruptions. When you use allocation strategy, EMR picks the most available Spot pools from the list of selected instance types. By adding more instance types to the list, you allow EMR to consider more, and potentially more available, pools to provision capacity from. When interruptions do occur, EMR is able to replace that capacity faster as it can choose from a wider pool of instances.
For large clusters, you can get better access to On-Demand capacity by specifying as many instance types as possible. In this case, EMR is able to pick from a wider pool of instances and provision capacity across instance types until your requirements are satisfied.
To get started, read Configure Instance Fleets and the EMR API Reference.