The Ec2Resource object in AWS Data Pipeline is failing because of the following error. How do I resolve this?

Resource is stalled. Associated tasks not able to make progress.

Data Pipeline gives Amazon Elastic Compute Cloud (Amazon EC2) instances a set amount of time to start. If an EC2 instance takes too long to start, the Ec2Resource object fails because Data Pipeline assumes that the instance isn't ready.

Here are some common reasons why Amazon EC2 instances time out in Data Pipeline.

Software updates after launch

If you don't specify the optional imageId field for the Ec2Resource object, Data Pipeline defaults to an older Amazon Machine Image (AMI). Instances launched from older AMIs automatically install software updates after they launch, and they might time out in Data Pipeline.

Use one of the following options to resolve this issue:

  • Specify the latest AMI in the imageId field for the Ec2Resource object. This significantly reduces the amount of time required for software updates.
  • Use an existing resource instead of the Ec2Resource object. Install Task Runner on an existing EC2 instance, and then add a workerGroup field to connect the object to the pipeline activities that it should process. For more information, see Executing Work on Existing Resources Using Task Runner.

Network connectivity

Task Runner must be able to access the AWS Data Pipeline endpoint and Amazon Simple Storage Service (Amazon S3). This means that the virtual private cloud (VPC) that the EC2 instance is in must have a subnet and an internet gateway.

Keep the following subnet requirements in mind:

For more information, see Launching Resources for Your Pipeline into a VPC.

Task Runner installation failure

When you use the EC2Resource object, cloud-init installs Task Runner. If installation fails, the Amazon EC2 instance can't communicate with the Data Pipeline endpoint. For example, if the pipelineLogUri field contains unexpected characters, such as quotation marks, the Task Runner installation script fails. Check the cloud-init logs for the EC2 instance to find out why the installation failed.

Note: After you change a pipeline definition, you must reactivate the pipeline for your changes to take effect. For more information, see Editing Your Pipeline.

Missing AWS Identity and Access Management (IAM) permission

The IAM instance profile used by the EC2 instance must give Task Runner permission for the APIs that Task Runner attempts to call, such as PollForTask. Connect to the EC2 instance and then check the Task Runner logs to find out if a missing permission is causing the error.

Did this page help you? Yes | No

Back to the AWS Support Knowledge Center

Need help? Visit the AWS Support Center

Published: 2018-08-03