I set up automatic scaling for my Amazon EMR nodes, but the nodes are scaling down slowly. What is causing this, and how can I fix it?

Check for these common causes of slow scale-downs:

Instance group in an arrested state

Check if the instance group is in an arrested state. This state occurs when there are too many errors while launching cluster nodes. For potential causes of node-launch failure, see Check for Arrested Groups.

After you resolve the issues with launching nodes, reset the number of nodes you want on the cluster's instance group. The instance group then resumes allocating nodes. Use the AWS Command Line Interface (AWS CLI) to reset a cluster in an arrested state.

Note: Running nodes aren't restarted or terminated in an arrested state.

Scale-down activity or cooldown period in progress

Check if scale-down activity is already in progress. If a prior scale-down is complete, check if the cooldown period is expired. After the cooldown period for a prior scaling policy expires, further scaling activity is performed as needed.

For more information on the cooldown period, see Scaling Cooldowns.

Scale-down behavior configured on the EMR cluster

Check the scale-down behavior configured on your EMR cluster. The "Terminate at task completion" scale-down behavior terminates instances after the task completes. The "Terminate at instance hour" scale-down behavior terminates instances one full hour after the task completes.

To reduce the time to scale down, consider using the "Terminate at task completion" setting. For more information on scale-down behavior, see Configuring Amazon EMR Scale-Down Behavior.

Note: If your nodes aren't scaling at all, see Why aren't my EMR nodes scaling?

Did this page help you? Yes | No

Back to the AWS Support Knowledge Center

Need help? Visit the AWS Support Center

Published: 2018-05-23