How can I increase Amazon EBS volumes in AWS Batch using a launch template?

Last updated: 2020-02-03

I want to increase Amazon Elastic Block Store (Amazon EBS) volumes in AWS Batch using a launch template.

Short Description

You can modify the storage configuration on your instances created in AWS Batch without using a custom Amazon Machine Image (AMI).

To increase Amazon EBS volumes in AWS Batch, you must create a launch template before associating that template with your compute environment.

You can create a launch template with the following:

  • Amazon Elastic Compute Cloud (Amazon EC2) console
  • AWS Command Line Interface (AWS CLI)

The following resolution shows you how to create a launch template with the AWS CLI.

Note: The DeviceName on the launch template can differ depending on the Amazon Elastic Container Service (Amazon ECS)-optimized AMI version that you're using. By default, AWS Batch is based on Amazon Linux 1, which comes with two Amazon EBS volumes (/dev/xvda and /dev/xvdcz). If you're using Amazon Linux 2, AWS Batch has only a single volume (/dev/xvda). For more information, see AMI Storage Configuration.

Important: To increase your storage limit beyond the Docker storage limit (10 GiB per container) for Amazon Linux 1, see How do I resolve the "No space left on device" error in AWS Batch?

Resolution

Create a launch template

1.    Create a file named launch-template-data.json to resize the volumes for your compute environment. See the following examples.

Example of template to increase volumes for Amazon Linux 1 (/dev/xvda) and (/dev/xvdcz):

{
    "LaunchTemplateName":"increase-all-volume",
    "LaunchTemplateData": {
        "BlockDeviceMappings": [
          {
            "Ebs": {
              "DeleteOnTermination": true,
              "VolumeSize": 50,
              "VolumeType": "gp2"
            },
            "DeviceName": "/dev/xvda"
          },
          {
            "Ebs": {
              "DeleteOnTermination": true,
              "VolumeSize": 75,
              "VolumeType": "gp2"
            },
            "DeviceName": "/dev/xvdcz"
          }          
        ]
  }
}

Example of template to increase volume for Amazon Linux 2 (/dev/xvda):

{
    "LaunchTemplateName": "increase-volume",
    "LaunchTemplateData": {
        "BlockDeviceMappings": [
            {
                "DeviceName": "/dev/xvda",
                "Ebs": {
                    "VolumeSize": 100,
                    "VolumeType": "gp2"
                }
            }
        ]
    }
}

2.    To create a launch template, run the following command:

aws ec2 --region us-east-1 create-launch-template --cli-input-json file://launch-template-data.json

Note: Replace us-east-1 with your AWS Region.

The output should be similar to the following:

{
    "LaunchTemplate": {
        "LaunchTemplateId": "lt-06935eb650e40f886",
        "LaunchTemplateName": "increase-volume",
        "CreateTime": "2019-12-26T09:40:46.000Z",
        "CreatedBy": "arn:aws:iam::12345678999:user/alice",
        "DefaultVersionNumber": 1,
        "LatestVersionNumber": 1
    }
}

Create a new compute environment to associate with your new launch template

1.    Open the AWS Batch console.

2.    In the navigation pane, choose Compute environments.

3.    Choose Create environment.

4.    In the Configure your Compute resources section, for Launch Template, select the launch template that you created earlier.

5.    Choose Create.

Note: The instances show the new drive sizes when AWS Batch spins up the instances.

6.    To test if the volumes are resized, use SSH to connect to the instance launched by AWS Batch, and then run the following command:

$ lsblk

The output should be similar to the following example:

NAME                                                                          MAJ:MIN   RM  SIZE RO TYPE MOUNTPOINT
xvda                                                                          202:0      0   50G  0 disk 
`-xvda1                                                                       202:1      0   50G  0 part /
xvdcz                                                                         202:26368  0   75G  0 disk 
`-xvdcz1                                                                      202:26369  0   75G  0 part 
  |-docker-docker--pool_tdata                                                 253:1      0 74.1G  0 lvm  
  | `-docker-docker--pool                                                     253:2      0 74.1G  0 lvm  
  |   `-docker-202:1-394378-b6aedb5f6f6fd8af3731edb7043bafa704d9ff9752495f8eb24e613ec5dd85fd
                                                                              253:3      0   10G  0 dm   /var/lib/docker/devicemapper/mnt/b6aedb5f6f6fd8a
  `-docker-docker--pool_tmeta                                                 253:0      0   80M  0 lvm  
    `-docker-docker--pool                                                     253:2      0 74.1G  0 lvm  
      `-docker-202:1-394378-b6aedb5f6f6fd8af3731edb7043bafa704d9ff9752495f8eb24e613ec5dd85fd
                                                                              253:3      0   10G  0 dm   /var/lib/docker/devicemapper/mnt/b6aedb5f6f6fd8a

The output shows that /dev/xvda is 50G and /dev/xvdcz is 75G.


Did this article help you?

Anything we could improve?


Need more help?