How do I troubleshoot Amazon ECS tasks for Fargate that are stuck in the Pending state?

4 minute read
0

My Amazon Elastic Container Service (Amazon ECS) task that's running on AWS Fargate is stuck in the PENDING state.

Resolution

Check what routes to the internet your subnets are using

For Fargate tasks in a public subnet:

Verify that your Fargate task has an assigned public IP address and a default route (0.0.0.0/0) to an internet gateway. To do this, select the Enable auto-assign public IPv4 address check box when you launch your task or create a new service. For more information, see Public IPv4 addresses.

Note: You can't select the Enable auto-assign public IPv4 address check box for existing tasks or services.

For Fargate tasks in a private subnet:

Verify that your Fargate task has a default route (0.0.0.0/0) to either a NAT gateway, AWS PrivateLink, or another source of internet connectivity.

  • If you're using a NAT gateway, then place your NAT gateway in a public subnet. For more information, see Architecture with an internet gateway and a NAT gateway.
  • If you're using AWS PrivateLink, then confirm that your Fargate infrastructure is allowed to use security groups for your Amazon Virtual Private Cloud (Amazon VPC) endpoints.

Check your network access control list and security group settings

Verify that your network access control list (network ACL) and security groups don't block outbound access to port 443 from the subnet. For more information, see Control traffic to resources using security groups.

Note: Fargate tasks must have outbound access to port 443 to activate outgoing traffic and reach Amazon ECS endpoints.

Check your VPC endpoints

If you're using AWS PrivateLink, confirm that you have the required endpoints.

Required endpoints for Fargate platform versions 1.3.0 or earlier:

  • com.amazonaws.region.ecr.dkr
  • S3 gateway endpoint

Required endpoints for Fargate platform versions 1.4.0 or later:

  • com.amazonaws.region.ecr.dkr
  • com.amazonaws.region.ecr.api
  • S3 gateway endpoint

Note: If your task definition uses AWS Secrets Manager, SSM parameters, or Amazon CloudWatch Logs, you might need to define endpoints. For more information, see Using an AWS Secrets Manager VPC endpoint and Using CloudWatch Logs with interface VPC endpoints.

While using PrivateLink, confirm that the security groups for your VPC endpoints allow the Fargate infrastructure to use these security groups.

Check your AWS Identity and Access Management (IAM) roles and permissions

The task execution role grants the Amazon ECS container and Fargate agents permission to make AWS API calls on your behalf. This role is required by Fargate when you:

  • Pull a container image from Amazon Elastic Container Registry (Amazon ECR)
  • Use the awslogs log driver
  • Use private registry authentication
  • Reference sensitive data using Secrets Manager secrets or AWS Systems Manager Parameter Store parameters

If your use case involves any of the preceding scenarios, confirm that you have the right permissions defined in your task execution role. For a complete list of required permissions, see Amazon ECS task execution IAM role.

Check for issues with pulling the image

If you receive a cannotpullcontainer error for your Fargate task, then complete the steps in How do I resolve the "cannotpullcontainererror" error for my Amazon ECS tasks on Fargate?

VPC in dual stack mode

When using a VPC in dual-stack mode with Fargate, you can configure your VPC with an internet gateway or an outbound-only internet gateway for tasks that are assigned an IPv6 address to access the internet. For more information, see Using a VPC in dual-stack mode.

Note: To troubleshoot your issue, you can also use Amazon ECS Exec to retrieve the logs from the container instance of your task or service.

Container dependency is defined

A container dependency that's defined in the Task Definition can lead to the Fargate task to be in PENDING state indefinitely. Example: If containerA depends on certain state of containerB, containerA is expected to stay in PENDING state until the containerB reaches that state. But if containerB never reaches the desired state, then the task stays in PENDING state indefinitely. Be sure that you have the dependencies appropriately or evaluate the dependencies.

For more information, see Container dependency.


AWS OFFICIAL
AWS OFFICIALUpdated 2 years ago