How do I troubleshoot issues when setting up a new capacity provider or update an existing capacity provider for my Amazon ECS cluster?

Last updated: 2022-04-13

I'm having issues when I try to set up a new capacity provider or update an existing capacity provider for my Amazon Elastic Container Service (Amazon ECS) cluster.

Short description

Some of the most common issues that occur when you set up a new capacity provider or update an existing capacity provider for your Amazon ECS cluster are the following:

  • You get errors when you create an auto scaling group capacity provider using AWS Command Line Interface (AWS CLI).
  • You get the error: The specified capacity provider already exists. To change the configuration of an existing capacity provider, update the capacity provider.
  • You get the error: The specified Auto Scaling group ARN is already being used by another capacity provider. Specify a unique Auto Scaling group ARN and try again.
  • You get the error when you create the capacity provider strategy: The specified cluster is in a busy state. Cluster attachments must be in UPDATE_COMPLETE or UPDATE_FAILED state before they can be updated. Wait and try again.
  • You get errors because of some limitations with the capacity provider.
  • The capacity provider is created successfully with AWS CloudFormation, but not shown in the console.

Resolution

You get errors when you create an Auto Scaling group capacity provider using AWS CLI

To troubleshoot this issue when you create a capacity provider using AWS CLI, review AWS CloudTrail events to check for the CreateCapacityProvider API. The following are some of the errors that you might get with the CapacityProvider API:

  • ClientException: These errors are usually caused by a client action. This client action might be using an action or resource on behalf of an AWS Identity and Access Management (IAM) user that doesn't have the required permissions to use the action or resource. This error might also specify an identifier that isn't valid.
  • UpdateInProgressException: You get this error because there is already a current Amazon ECS container agent update in progress on the container instance that you specified. If the container agent becomes disconnected while it's in a transitional stage, such as PENDING or STAGING, the update process might get stuck in that state. However, when the agent reconnects, it resumes where it stopped previously.

For more information, see Errors.

Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.

You get the error: The specified capacity provider already exists. To change the configuration of an existing capacity provider, update the capacity provider

When you create the capacity provider, use a name that's different from any of the capacity provider names you used previously. Or, you can delete the existing capacity provider with that name and then create a new capacity provider.

To view a list of all capacity providers that you created, run the following command:

aws ecs describe-capacity-providers --query capacityProviders[].name --region example-region

The output looks similar to the following:

[
    "FARGATE",
    "FARGATE_SPOT",
    "ecs-capacityprovider-1"
]

To delete a capacity provider, run the following command:

aws ecs delete-capacity-provider --capacity-provider ecs-capacityprovider-1 --region example-region

For sample output and more options, see delete-capacity-provider.

To create a new capacity provider, run the following command:

aws ecs create-capacity-provider --name "example-capacity=provider" --auto-scaling-group-provider autoScalingGroupArn=arn:aws:autoscaling:us-east-1:123456789012:autoScalingGroup:a1b2c3d4-5678-90ab-cdef-EXAMPLE11111:autoScalingGroupName/MyAutoScalingGroup,managedScaling={status=ENABLED,targetCapacity=100,minimumScalingStepSize=1,maximumScalingStepSize=100},managedTerminationProtection=ENABLED

For sample output and more options, see create-capacity-provider.

You get the error: The specified Auto Scaling group ARN is already being used by another capacity provider. Specify a unique Auto Scaling group ARN and try again

The Auto Scaling group and capacity provider must have a one-on-one relationship. You can attach an Auto Scaling group to exactly one capacity provider and vice versa. Therefore, verify that your capacity provider is associated with only one Auto Scaling group.

You get the error: The specified cluster is in a busy state. Cluster attachments must be in UPDATE_COMPLETE or UPDATE_FAILED state before they can be updated. Wait and try again

This error auto resolves in a few minutes. You can wait for some time and then try again to update the capacity provider. You can also run the following CLI command to view the current status of the attachments, and wait until the attachment is in the UPDATE_COMPLETE status.

aws ecs describe-clusters --clusters example-ARN --include ATTACHMENTS --query clusters[].attachmentsStatus --region example-region

The output must look similar to the following:

[
    "UPDATE_COMPLETE"
]

You get errors because of some limitations with the capacity provider

Keep the following limitations in mind when you create or update a capacity provider:

  • You can't specify more than six capacity providers for a capacity strategy.
  • You can use either Auto Scaling group capacity providers or AWS Fargate capacity providers in your cluster strategy. You can't use a combination of both types.
  • The base value for the capacity provider strategy is supported only when you run the tasks.

The capacity provider is created successfully with CloudFormation, but not shown in the console

This might happen when you don't associate the capacity provider with the cluster in your CloudFormation template.

To avoid this issue, use the following snippet in your CloudFormation template to associate the capacity provider with the cluster:

...
ECSCluster:
  Type: 'AWS::ECS::Cluster'
  Properties:
    ClusterName: example-cluster
    CapacityProviders:
      - !Ref ECSCapacityProvider
...