Containers
Amazon ECS on AWS Fargate now enables customers to configure ephemeral storage up to 200GiB
Today, we are announcing support in AWS Fargate to configure ephemeral storage up to 200 GiB in size. Tens of thousands of customers use Amazon Elastic Container Service (Amazon ECS) with AWS Fargate to benefit from the serverless compute model for a wide variety of container-based applications at scale. As container adoption has grown, the use cases have gone beyond simply running microservices. The need for larger storage ranges from customers who run machine learning inference, use larger Docker images (greater than 20GiB), as well as data processing with large data sets in the container. Previous to this release, customers would have to choose EC2 as the compute option for their containers when running in Amazon ECS and Amazon Elastic Kubernetes Service (Amazon EKS). With this latest feature, Fargate will automatically provision and manage the storage with the specified size during the lifetime of your task.
To demo how we can set the storage for a Fargate task, we’ll deploy a container using the AWS Copilot CLI. We’ll set the size of our desired storage, and exec into the container and explore. Let’s get started!
Since we are deploying a simple demo, we will create a Dockerfile that defines how we want to package and run our application. At runtime, the container will simply sleep, giving us time to enter into the container to check out the storage.
At this point, we are ready to get this container deployed to a live environment! We’ll start by initializing our application, and answering some basic questions to define the structure of our application and service using the Copilot CLI.
We will wait to create our environment and deploy the service as we need to modify our manifest file to set the storage size of our Fargate tasks. For this demo, I’m going to set the size of the storage to the max size of 200GiB.
In the manifest file (located at ./present-directory/copilot/svc-name/manifest.yml), we simply add the storage parameter and will pass in the value of 200, which represents GiB. For more information, see the Copilot documentation.
At this point, we are ready to deploy our environment and test this out! Let’s create our environment.
Now let’s deploy our container!
Once our environment is deployed, we can connect to our container in an interactive shell using ECS Exec with Copilot. If you aren’t familiar with ECS Exec, we recently released this feature and you can read more about it in this blog post.
Now that we’re in, let’s check the disk space available to us in the container. I will do this by running a simple df command.
df -H
The result shows us that we have 200 GiB of storage available to the task, and that’s it!
In this demo, we deployed a Fargate task on Amazon ECS and set the ephemeral storage size to 200 GiB based on our requirements. We validated this by entering into a terminal on the container and inspecting the amount of disk space available. As mentioned in the beginning of the blog, this feature enables customers that want to take advantage of serverless functionality with Fargate but were previously constrained by the default ephemeral storage size of 20GiB. Aside from support in AWS Copilot, this feature is supported when deploying Fargate tasks via AWS CloudFormation as well as the AWS CLI. See the AWS Fargate documentation for more information. Finally, you can take advantage of this new feature on AWS Fargate Platform Version 1.4, or by using the LATEST platform version in your task definition.