Use AWS Systems Manager Automation runbooks to resolve Elastic Block Store related operational tasks

Customers have been using various forms of automation for years to define a sequence of actions on Amazon Elastic Block Store (EBS). While before, customers were facing operational overhead related to EBS tasks, AWS Systems Manager (SSM) Automations can now be leveraged to meet a wide variety of customer use cases. In this blog post, a deep dive into the “AWS-ExtendEbsVolume” runbook will be outlined to showcase the powerful capabilities of SSM Automation to increase efficiency. Now, operations engineers and IT professionals can use these runbooks to resolve operational tasks.

AWS SSM Automation simplifies common maintenance and deployment tasks for Amazon EBS and many other AWS Services. The option to create custom runbooks or use AWS managed predefined runbooks is available.

The following diagram shows a general architecture of a solution that launches an AWS Systems Manager Automation runbook.

            Figure 1: General Architectural of the Solution

Solution Overview

Customers can use SSM Documents to standardize EBS volume configurations, automate snapshot creation for backups, and dynamically modify volume properties based on performance and cost optimization requirements. This blog recognizes the customer need for automation and provides one example of leveraging and SSM runbook to meet customer use cases. This will outline the steps needed to extend EBS volumes at a high level and the process which is consistent across other SSM runbooks related to EBS.

Prerequisites:

Before proceeding, you need the following prerequisites:

• The Amazon Elastic Compute Cloud (EC2) instance must be managed by AWS SSM.
• The EC2 instance must be attached with an EBS volume as well.
• The operating systems types supported are Linux and Windows.
  • For Linux supported File Systems are xfs and ext4.
  • For Windows supported File System is NTFS, and ReFS.
• The new size of the drive must be less than the maximum supported size for that drive in case of windows.
• A role with enough permissions should be provided for the document to be able to start or stop the configuration recording (otherwise the document will run with the caller identity).
• AutomationAssumeRole is an Identity and Access Management (IAM) role for this execution of this document. If no role is specified, AWS SSM Automation will utilize your IAM permissions to execute this document. To know more about granting permissions by using IAM policies, refer here.

Launch with Linux:

Next, is an illustration of how the “AWS-ExtendEBSVolume” runbook works by increasing the size of the volume to a target size.

In the EC2 console, select a test instance to modify its volume size.

Figure 2: Linux EC2 Instance

Issue: The Volume size of the EC2 Linux instance for the root volume is 8 GiB. The use case is to increase the volume size to the Target size of 30 GiB without shutting down or rebooting the instance.

Step1: Checking Volume Size in the EC2 console:

The size of volume “vol-xxx” attached to EC2 instance “i-xxx” is 8 GiB.

Figure 3: EC2 Linux Instance Volume Size

The detailed information can be viewed from expanding the Storage section of the EC2 instance.

Figure 4: EC2 Instance Storage in the Console

As mentioned above, the instance must be managed by AWS SSM. One way of verifying whether the instance is being managed by AWS Systems Manager is by checking the connection to the EC2 instance using “Session Manager”.

Figure 5: Connect to EC2 Instance using Session Manager

Step2: The session is established with EC2 instance Id “i-xxx”. It shows the current time, current disk, partition and the file system size when these commands were run.

• date “+%T”
• sudo lsblk
• df -Th
  

Figure 6: Established Session with EC2 Instance

Step3: Now, let’s use “AWS-ExtendEbsVolume” Automation Document to extend the EBS Volume:

• Open the AWS SSM via AWS Management Console, and Type “Systems Manager”
• In the navigation pane, choose Automation > Choose Execute Automation

Figure 7: AWS Systems Manager Automation Window

• In the Choose runbook section, under Automation runbook, search for the document named, “AWS-ExtendEbsVolume”. Click on the document to execute it.

Figure 8: Search the AWS SSM Automation runbook

• Click on Execute automation to run the automation workflow.

Figure 9: Execute Automation Window

Step 4: Please provide the Volume ID of the EC2 Instance (Required), SizeGib (Required), MountPoint (Required), InstanceId (Required) and then click the “Execute” button to start the Automation workflow:

Figure 10: Input Parameters for Linux

Here is some Information about Input Parameters:

Mandatory/Required:

VolumeId: ID of the volume which you want to extend. You will specify this ID in the procedure.
SizeGib: Size Gib is the target size of the volume, in Gib of the same volume for which you provided the Volume ID. The target volume size must be greater than or equal to the existing size of the volume.
MountPoint: (Required for Linux) Mount point (a directory or file at which a new file system, directory, or file is made accessible) such as “/”, “/data” etc. of the partition which is to be increased.
DriveLetter: (Required for Windows) Drive Letter for Windows is needed to increase the size of the existing volume.
InstanceId: The ID of the Amazon EC2 instance that the Amazon EBS volume you want to extend is attached to.

Optional:

keepSnapShot: Keep Snapshot is a Boolean flag to determine whether to remove the created snapshot after successful resizing of the volume and the file system. It is set to “true” by default.
AutomationAssumeRole: The IAM role for this execution. If no role is specified, AWS Systems Manager Automation will utilize your IAM permissions to execute this document. To know more about granting permissions by using IAM policies, refer here.

Step5: Now, let’s proceed to execute:

• Once you click the “Execute” button, the automation will start, and you can see the status as “In Progress”.
• See more details under executed steps.
• You can always check the details of each “Execution ID” and its associated steps using “Step ID”.

Figure 11: Overall Status of the Executed Runbook

Step6: Observations:

You can see the Execution “Overall status” under automation is “Success”.

Figure 12: Overall Success Status of the Executed Runbook

Step7: Outputs:

You can check the “Outputs” of the “Execution ID”.

Figure 13: Output Payload in the Outputs Section

Step8:

• Now, let’s check if the size of volume “vol-xxx” has increased through EC2 console.

Figure 14: Volume Size View in the EC2 Console After the SSM Runbook Execution

• Let’s check it through the EC2 instance “i-xxx” using AWS Session Manager. To get to this view follow the steps below:

1. 1. Open the AWS Management Console and navigate to the EC2 service.
   2. Locate the specific EC2 instance “i-xxx” you want to access.
   3. Look for the “Connect” button or dropdown menu associated with the instance.
   4. Select the “Session Manager” option from the connect menu.

• • The session is established with EC2 instance Id “i-xxx”. It shows before and after extending the volume and the file system when these commands were run.
    • date “+%T”
    • sudo lsblk
    • df -Th

Figure 15: Before and After Extending the Volume and the File System

The disk and the partition sizes have increased to 30G and the File system /dev/xvda1 has also increased to 30G.

Launch with Windows:

Next, “AWS-ExtendEBSVolume” runbook works by increasing the size of volume to a target size in Windows.

In EC2 console, select a test instance to modify its volume size.

Figure 1: Windows EC2 Instance

Issue: The Volume size of the EC2 Windows instance for the C is 30 GiB. I need to increase the volume size to the Target size of 40 GiB without shutting down or rebooting the instance.

• As you can see the C: NTFS volume size is 30GiB.

Figure 2: Drive C with 30GB of NTFS File System

• Follow the above mentioned steps (1, and 3-7) as these are same for Windows instance as well.
• Please provide the Volume ID of the EC2 Instance (Required), SizeGib (Required), DriveLetter (Required), InstanceId (Required) and then click the “Execute” button to start the Automation workflow:

Figure 3: Input Parameters for Windows

• The “AWS-ExtendEbsVolume” for windows instance is being executed.

Figure 4: Execution Overall In Progress Status

• The execution has been successful. The below picture shows the Output payload as an output.

Figure 5: Output Payload in the Outputs Section

• The volume size has increased to 40G and the File system NTFS has also increased to 40G.

Figure 6: Drive C with 40GB of NTFS File System

• Here is another example of Windows ReFS file system before increasing the volume from 100 GiB to 120 GiB using the “AWS-ExtendEBSVolume” runbook.

Figure 7: Drive D Before Extending the Volume and the ReFS File System

• Increased volume size after running the “AWS-ExtendEBSVolume” runbook.

Figure 8: Drive D After Extending the Volume and the ReFS File System

Figure 9: Both Increased Volumes View Through EC2 Console

Do not extend the volume if it is in the In-user – Optimizing (%) state because if you execute the automation document on this volume then the automation will fail with the error message that “ClientError – An error occurred (IncorrectModificationState) when calling the ModifyVolume operation: Volume vol-xxxxxxxxxxxxxxxxx cannot be modified in modification state OPTIMIZING”.

Figure 10: Client Error with Incorrect Modification State

Do not use a new size smaller than the existing drive size as this will fail because it is not supported to decrease the volume size. The execution of the document will fail with the error message “ClientError – An error occurred (InvalidParameterValue) when calling the ModifyVolume operation: New size cannot be smaller than existing size”.

Figure 11: Client Error with Invalid Parameter Value

Conclusion

In this post, it shows ways to use various AWS managed predefined runbooks for Amazon EBS volumes using Amazon SSM Automation for operational tasks. The “AWS-ExtendEBSVolume” runbook increases the size of an Amazon EBS volume and extends the file system, all without provisioning additional automation infrastructure or resources.

The “AWS-ExtendEBSVolume” runbook ran in AWS Account using the AWS console, but you can also use Automation runbooks as standalone, through other AWS services, such as AWS Service Catalog,  Amazon EventBridge, AWS CloudFormation or third party products.

SSM is available in all major AWS Regions. For any questions about this blog post, contact AWS Support.

About the Authors