Containers

Announcing Amazon ECS Task Definition Deletion

Today, we are happy to announce new functionality in Amazon Elastic Container Services (Amazon ECS) that allows you to delete task definition revisions. Until now, you were only able to deregister a task definition revision and it would no longer display in your ListTaskDefinition API calls or in your Amazon ECS console, unless you specifically chose to select task definitions revisions that were INACTIVE. With this new capability, customers can permanently delete task definition revisions that are no longer needed or contain undesirable configurations, which simplifies resource management and improves security posture.

Introduction

A task definition serves as the blueprint for running tasks and services on Amazon ECS. Customers can update task definitions to create new revisions, and deregister old revisions that are no longer needed. Deregistered task definition revisions are marked INACTIVE by Amazon ECS and cannot be used for creating new services or running standalone tasks. While deregistration makes it easier for customers to disregard INACTIVE task definition revisions, these resources could pile up over time. Our customers gave us feedback that managing task definitions was problematic due to reasons:

  • Sensitive information (e.g., secrets that could be deleted and were exposed in text format)
  • Large number of task definition revisions were cluttering resource counts

Walkthrough

How to use the new delete task definition functionality

Starting today, you can now use the new DeleteTaskDefinition API to delete your tasks revisions and delete INACTIVE revisions as well. You can do this using the AWS Command Line Interface (AWS CLI), AWS Cloud Development Kit (AWS CDK) and the AWS SDK’s that you are familiar with. To do this in the AWS Console, please see the instructions here

Amazon ECS enables you to deregister task definitions that you don’t want to use to launch new tasks or services. You can do so by using the DeregisterTaskDefinition API call.

Run the following command in your AWS CLI to list all the task definitions that have a state of INACTIVE.

aws ecs list-task-definitions --status INACTIVE --no-cli-pager

If you have an active task revision that you need to deregister, then run the following command:

aws ecs deregister-task-definition --task-definition <task name:revision>

You can use the AWS Console to deregister a single task definition revision. If you have hundreds or thousands revisions to deregister, then it would be better to use a script to batch these operations.

The following script can be used to manage these operations at scale. The script requires three variables: TASKNAME, START, and END. TASKNAME is the name of the task definition family to deregister. START is the first task definition revision that you would like the deregistration loop to start at. END is the last task definition revision you want the loop to stop.  For example, you can use the script to delete all task revisions between 1624 and 1854 by setting START=1624 and END=1854, or you can delete the first 1000 task definition revisions by setting START=1 and END=1000

The script executes a loop and iterates through the task definition revisions, deregistering them with a sleep of 5 seconds in between, to ensure that you don’t encounter any API throttling issues.

#!/bin/bash -ex

TASKNAME=<task name>
START=1 # the first number of the task revision to loop through
END=1000 # The last number to stop the delete loop at

for (( x=$START; x<=$END; x++ ))
do
        aws ecs deregister-task-definition --task-definition $TASKNAME:$x --no-cli-pager
        sleep 5
        echo "The task $TASKNAME and revision $x has been deregistered"
done

Let’s look at how you would use the AWS CLI to delete a single task definition revision.

aws ecs delete-task-definitions --task-definitions <task name:revision>

Below we have a script that automates the delete task definition revision operation, it uses the same logic as the deregister script with the same variables TASKNAME,START and END.

#!/bin/bash -ex

TASKNAME=<task name>
START=1 # the first number of the task revision to loop through
END=1000 # The last number to stop the delete loop at

for (( y=$START; y<=$END; x++ ))
do
        aws ecs delete-task-definitions --task-definitions $TASKNAME:$y --no-cli-pager
        sleep 5
        echo "The task $TASKNAME and revision $x has been deleted"
done

Once a task definition revision has been deleted, it transitions from the INACTIVE state to DELETE_IN_PROGRESS. To see this in action, we use the describeTaskDefintion API or run the following command.

aws ecs describe-task-definition --task-definition <task name:revision>

You can use the following script to also combine both the deregister and delete operations into a single script.

#!/bin/bash -ex

TASKNAME=<task name>
START=1 # the first number of the task revision to loop through
END=1000 # The last number to stop the delete loop at

# This function will deregister the task definition
for (( x=$START; x<=$END; x++ ))
do
        aws ecs deregister-task-definition --task-definition $TASKNAME:$x --no-cli-pager
        sleep 5
        echo "The task $TASKNAME and revision $x has been deregistered"
done

# This function will delete the task definition
for (( y=$START; y<=$END; y++ ))
do
        aws ecs delete-task-definitions --task-definitions $TASKNAME:$y --no-cli-pager
        sleep 5
        echo "The task $TASKNAME and revision $y has been deleted"
done

To track the deletion status, you can use the following command to retrieve a list of task definition revisions marked for deletion:

aws ecs list-task-definitions --status DELETE_IN_PROGRESS --no-cli-pager

During our testing it took between 30-60 minutes for the DELETE_IN_PROGRESS process to complete using the above script. ECS task definitions are crucial resources that could be referenced by running tasks as well as by various ECS components to provide high availability and faster performance to customers. ECS, therefore, performs deletion of task definition revisions asynchronously to make sure that your ECS services continue to operate and meet their availability criteria without any disruptions.

Prerequisites

  • AWS account
  • AWS CLI

Conclusion

In this post, we showed how customers can permanently delete task definition revisions that are no longer needed or contain undesirable configurations, which simplifies resource management and improves security posture. We walked you through examples of deregistering and deleting a single task definition revision with the AWS CLI, and shared a batch deletion script for customers that want to delete multiple task definition revisions.

Task definition deletion has been one of the most requested ECS features on our GitHub roadmap and we are excited to bring this to you. In the future, Amazon ECS will be delivering smarter and more automated mechanisms for customers to manage their task definition resources at scale. We plan to introduce task definition lifecycle policies, which customers can use to define rules to automate deregistration and deletion of their task definition resources. For instance, customers will be able to define a rule to automatically deregister a task definition revision that has not been in use for more than 180 days, and delete any task definition revision that has been unused and inactive for more than 90 days. We look forward to hearing your feedback on the Github issue