I want to implement Auto Scaling on an Amazon EMR cluster. How do I do that?

When automatic scaling is configured, Amazon EMR adds and removes instances based on Amazon CloudWatch metrics that you specify. The following are two of the most common metrics used for automatic scaling in Amazon EMR:

  • YarnMemoryAvailablePercentage: The percentage of remaining memory available to YARN.
  • ContainerPendingRatio: The ratio of pending containers to containers allocated. You can use this metric to scale a cluster based on container-allocation behavior for varied loads, which is useful for performance tuning.

For more information about choosing instance types and the amount of storage to provision for each node type, see Cluster Configuration Guidelines.

  1. Follow the steps at Using Automatic Scaling in Amazon EMR.
  2. To confirm that the Auto Scaling policy is attached to the instance group, choose Events from the navigation pane and check for Automatic Scaling Policy Events.

Did this page help you? Yes | No

Back to the AWS Support Knowledge Center

Need help? Visit the AWS Support Center

Published: 2018-08-31