How do I troubleshoot issues related to scheduled tasks in Amazon ECS?

Last updated: 2022-03-31

I have scheduled my Amazon Elastic Container Service (Amazon ECS) task to run periodically. However, my Amazon ECS task isn't triggered. I'm not getting execution logs or the history of the tasks in the cluster.

Resolution

When you use a scheduled Amazon ECS task, Amazon CloudWatch Events calls the RunTask API to Amazon ECS to run the tasks on your behalf.

Your scheduled Amazon ECS task might not be invoked due to the following reasons:

  • The Amazon EventBridge time or cron expression is configured incorrectly.
  • The EventBridge rule doesn't invoke the target.
  • The RunTask API failed to run.
  • The container exit due to application issues or resource constraints.

Check whether the EventBridge cron expression is configured incorrectly

To get the EventBridge cron expression, run the following AWS Command Line Interface (AWS CLI) command:

$ aws events describe-rule --name "example-rule" --region example-region

In the output of the command, you can view the configured EventBridge cron expression in the parameter ScheduleExpression. Be sure that you set the schedule for the rule in UTC+0 time zone.

Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.

Check whether the rule doesn't invoke the target

Use the Amazon CloudWatch metrics generated by EventBridge to view the rule performance. Invocation datapoints indicate that the target was invoked by the rule. If FailedInvocations data points are present, then there is an issue invoking the target. FailedInvocations represent a permanent failure and might be the result of incorrect permissions or a misconfiguration of the target.

To review the CloudWatch metrics for the EventBridge rule, do the following:

  1. Open the CloudWatch console.
  2. In the navigation pane, choose Metrics, and then choose All metrics.
  3. Choose Events.
  4. Choose By Rule Name.
  5. Select the TriggerRules, Invocations, and FailedInvocations (if available) metrics for the EventBridge rule that's configured to run the ECS task.
  6. Choose the Graphed metrics tab.
  7. For all metrics listed, select SUM for Statistic.

If FailedInvocations data points are present, there might be an issue related to inadequate target permissions. Be sure that EventBridge has access to invoke your ECS task. Verify that the EventBridge AWS Identity and Access Management (IAM) role has the required permissions. For more information, see Amazon ECS CloudWatch Events IAM Role.

Check whether the RunTask action failed to run

To verify if the RunTask API failed to run, search in AWS CloudTrail event history for RunTask within the time range of when the scheduled ECS task was expected to be invoked.

To find if the scheduled task wasn't invoked because the RunTask action failed, do the following:

  1. Open the AWS CloudTrail console.
  2. In the navigation pane, choose Event history.
  3. In the Event history page, for Lookup attributes, select Event name.
  4. For Enter an event name, enter RunTask.
  5. Choose the time range in the time range filter based on when the scheduled ECS task was expected to run.
    Note: The preset values for time range are 30 minutes, 1 hour, 3 hours, and 12 hours. To specify a custom time range, choose Custom.
  6. From the results list, choose the event that you want to view.
  7. Scroll to Event record on the Details page to view the JSON event record.
  8. Look for errorMessage or responseElements.failures.reason elements in the JSON event record.
    These elements in the JSON event record display the reason for the scheduled ECS task not being invoked.

For examples of RunTask API failure reasons and their causes, see API failure reasons.

Check whether the container exited after the task ran

The Amazon ECS tasks might be stopped even after the task runs successfully due to application issues or resource constraints. For more information, see How do I troubleshoot issues with containers exiting in my Amazon ECS tasks?


Did this article help?


Do you need billing or technical support?