I set up automatic scaling for my Amazon EMR nodes, but the nodes are not scaling. What is causing this, and how can I fix it?

Check for these common causes of unsuccessful scaling:

Instance group at capacity

Check if the instance group is already at the minimum or maximum capacity specified in the automatic scaling rule. You specify the minimum and maximum number of instances when you create automatic scaling rules for the cluster. For more information on automatic scaling rules, see Understanding Automatic Scaling Rules.

Termination protection

If you enable termination protection on the Amazon EMR cluster, no Amazon Elastic Compute Cloud (Amazon EC2) instances are terminated in response to scaling activity.

For more information on termination protection, see Managing Cluster Termination.

IAM policy

If you're using custom AWS Identity and Access Management (IAM) policies, verify that you have the correct permissions to perform scaling activity in your Amazon EMR role, EMR-EC2 role, and EMR-automatic scaling role. For an example of how to use inline policies for these roles, see Use Inline Policies for User Permissions.

For more information on IAM policies for automatic scaling in Amazon EMR, see Configure IAM Roles for Amazon EMR Permissions to AWS Services, Creating the IAM Role for Automatic Scaling, and Using the AWS CLI to Configure Automatic Scaling.

EC2 usage limit

Check if Amazon EMR returned an "EC2 QUOTA EXCEEDED" error. This error means that you reached your EC2 usage limit. You can submit a Service Limit Increase request for your EC2 instances.

Note: If your nodes are scaling down slowly, see Why are my EMR nodes scaling down slowly?

Did this page help you? Yes | No

Back to the AWS Support Knowledge Center

Need help? Visit the AWS Support Center

Published: 2018-04-10

Updated: 2018-05-22