AWS Compute Blog
Dynamically reconfigure your AWS Outposts capacity using Capacity Tasks
This post is written by Brianna Rosentrater, Hybrid Edge Specialist SA and Adam Duffield, Senior Technical Account Manager.
AWS Outposts extends AWS infrastructure, AWS services, APIs, and tools to on-premises locations for workloads that require low latency, local data processing, or data residency. Outposts comes in a variety of form factors, from 42U Outposts racks to 1U and 2U Outposts servers. Outposts now supports self-service capacity management, making it easy for you to view and manage compute capacity on your Outposts. A default capacity configuration for each new Outpost is determined during the ordering process. This default configuration can subsequently be modified to create a range of instance sizes and quantities to meet your changing business needs. To do so, you create a capacity task, specify the instance sizes and quantity, and run the capacity task to implement the changes. This post focuses on how to use capacity tasks to perform multi-host reconfigurations and view existing capacity configurations.
Overview
Amazon Elastic Compute Cloud (Amazon EC2) capacity on an Outpost is determined by the total volume of compute capacity within the Outpost when ordered. Outposts can also be scaled up or out as needed during your commitment term. For further details on Outpost capacity planning including best practices, refer to the Capacity Planning – AWS Outposts High Availability Design and Architecture whitepaper. We recommend planning spare capacity for N+M host availability when making modifications to your Outpost capacity configuration if your workloads need to be highly available. To calculate, take the number of hosts (N) you need to run all your workloads, and then add (M) additional hosts to meet your requirements for server availability during failure and maintenance events.
View existing Outposts capacity configuration
Outposts users now have visibility into capacity configurations at both an instance family and host level. This gives greater insight into capacity usage and instance placements. Within the Outposts console, after choosing the Outpost ID on which you want to view the capacity configuration, two new views have been provided: the Instance view and the Rack view.
Instance view
The Instance view provides a granular breakdown of the currently deployed instances on the Outpost along with an overall view of instance family capacity pools and their usage, as shown in the following figure. The Instances section gives detailed information around the deployed instances, their associated instance ID, instance size, AWS managed service (if applicable), and asset ID of where the instance is running.
Figure 1 – Outposts Instance View
The Instance capacity distribution summary displays how the various instance sizes are allocated within each instance family, as shown in the following figure. Each host of the same instance family contributes its capacity to the overall pool, which is represented in this section as a percentage rather than number of instance slots. This shows the configured capacity, but it doesn’t reflect any level of usage.
Figure 2 – Outposts Instance Capacity Distribution Summary
The Instance capacity distribution details section, shown in the following figure, provides a more detailed breakdown of each instance family capacity pool. This section provides a view of the total available instance capacity, the number of used instances, and the number of instances that are unavailable to you at that time (such as when a hardware failure occurs).
Figure 3 – Instance Capacity Distribution Details
Rack view
The Rack view tab provides a more granular view of the overall configuration of each host on a per rack basis, as shown in the following figure. It allows you to analyze the spread and usage of the instance size allocations across each host (asset) and, when choosing the show instance details button, provides the instance ID of each used slot. Using the search box, you can filter by Instance Family or Instance Size to provide a more concise view. If you’re using Outposts server, the Rack view tab will show the capacity configuration of your server.
Figure 4 – Rack View
Obtaining a view of current configuration
Alongside these views, two buttons are available on each of the pages. The Export JSON button gives the ability to download a JSON formatted copy of the current configuration for an Outpost. This is especially useful if you’re looking to record current state, or wanting to use the JSON upload option when submitting a new capacity task. The JSON file structure provides the overall configuration of each capacity pool. However, it doesn’t provide any details in terms of usage. The second button, Modify Instance Capacity, provides a shortcut to creating a capacity task.
This level of capacity visibility is also now available through AWS Command Line Interface (AWS CLI)/Outposts API calls, which some may prefer over the console. For example, the list-assets
CLI command can be used to obtain a breakdown of the capacity configuration of each Outpost host:
aws outposts list-assets --outpost-identifier outpost-arn
Figure 5 – list-assets CLI command sample output
If you want to obtain details of running instances on an Outpost, such as the instance size, the asset ID on which an instance is running, and the related AWS service name (if relevant), then the list-asset-instances
CLI command can be used:
aws outposts list-asset-instances --outpost-identifier outpost-arn
Figure 6 – list-asset-instances CLI command sample output
The list-asset-instances
CLI command also allows you to filter through numerous dimensions, such as instance type or AWS service. For example, this can be particularly useful for quickly identifying all running instances of a certain type, such as the m5d.large instance type by using the following command:
aws outposts list-asset-instances --outpost-identifier outpost-arn --instance-type-filter m5d.large
Modifying the Outposts capacity configuration
Due to the finite nature of Outposts capacity, changing operational requirements often mean that adjustments need to be made to your Outposts capacity configuration over time as new workloads are identified or applications need scaling.
From the Outposts console page, choosing Capacity Tasks from the left-hand menu gives a list of previously run capacity tasks and their status. From here, you can choose Create Capacity Task to start the process. To create a capacity task there are two options available: using an interactive capacity configuration tool through the Modify an Outpost capacity configuration option, or uploading a JSON file containing the necessary configuration through the Upload a capacity configuration option.
Figure 7 – Capacity Tasks web form
The interactive Modify an Outpost capacity configuration option using the simple Auto-balance feature and UI is the easiest way for those unfamiliar with Outpost capacity management to get started with making changes. Using this option, you can also choose one of two methods for the task:
- Run once: This results in the capacity task attempting to run a single time. If any instances block the successful application of the configuration, then the task fails.
- Run periodically over 48 hours or less: In the event of blocking instances, the capacity task is paused until the instances are stopped. The task rechecks the status every 10 minutes until it can run. If instances aren’t stopped within 48 hours, then the task is cancelled.
To build out the capacity task, capacity pools are displayed, grouped by instance families, and automatically populated with the current configuration of instance sizes and corresponding vCPU allocation. From here, you can make necessary changes to the existing capacity. Specifically, you can add new instance sizes and amend instance quantities in the corresponding fields. The total vCPU count for each instance family will update automatically to reflect your changes. The Auto-balance allows you to automatically adjust the quantities of individual instance sizes to fit within the total vCPU capacity available for the host, which is reflected for each capacity pool at the end. In the event that a capacity pool is over- or under-used, a warning is displayed. You can under provision a host if you choose, but the unprovisioned capacity is unusable, and attempting to overprovision the host results in the capacity task failing to run.
Figure 8 – Modifying existing capacity configuration
When the necessary changes have been made to the capacity pools, the second part of the capacity task configuration is choosing instances that should not be impacted by the running of the capacity task. During the run, you may not be in a position to stop certain instances, such as databases or AWS managed services such as Elastic Load Balancing (ELB) or Amazon ElastiCache, due to the impact to production workloads. Choosing these instances allows the capacity task to automatically try to find a path that avoids impacting them. However, in some situations capacity tasks may fail if the chosen instances block the successful running of the task, and there is no possible solution that avoids all the chosen instances. For example, if a capacity task was to remove all c5.xlarge instances and an instance was chosen to ‘keep as-is’ that was running on this instance size, then the task would fail. To avoid this, make sure to include these instances in your capacity configuration. For example, if you have five critical m5.4xlarge instances that must remain running, then include 5 m5.4xlarge instances in your m5 capacity pool configuration.
Figure 9 – Instances to keep as-is
After configuring the capacity pools and choosing the necessary instances to keep as-is, an overview of the changes is presented allowing you to validate the configuration prior to running. When you have reviewed the summary, select Create Task to trigger the execution of the capacity task using the chosen method. You can observe the status of a capacity task by choosing the capacity task ID. When it’s initially submitted, the status shows as Requested. During this time, the capacity task evaluates the necessary changes to determine if the task can proceed or if instances need stopping. If the Run once option was chosen and instances do need stopping, then the task moves to a cancelled status and provides details of the blocking instances. Alternatively, if Run periodically was chosen, the task remains in the Requested status until the listed instances have been stopped. In the event that the blocking instances can’t be stopped, the capacity task is cancelled after 48 hours. While a task is running, the Outpost hosts that are impacted by the configuration changes are placed into an isolated state. This means that capacity for new instance launches may be impacted. This isolation only lasts a few minutes while the capacity task is running, but it may impact auto scaling groups if a capacity task coincides with a scaling event.
Instead of using the interactive capacity configurator UI, you can also choose to upload a JSON file to the console containing the necessary configuration. Using this method, choosing instances to keep as-is isn’t available, and the method is automatically chosen as Run once. When a capacity task JSON file is uploaded, the resulting plan is displayed in the following text box and can be amended if needed. Alternatively, rather than uploading the file, the contents can be directly pasted into the text box. Choosing Next moves to the review screen where the remainder of the process continues in line with using the interactive capacity configurator UI.
Figure 10 – Upload a Capacity Configuration using JSON
You may also prefer using the AWS CLI/Outposts API for creating capacity tasks, and a number of new CLI/API actions are now available to support this:
cancel-capacity-task
/ CancelCapacityTaskget-capacity-task
/ GetCapacityTasklist-blocking-instances-for-capacity-task
/ ListBlockingInstancesForCapacityTasklist-capacity-tasks
/ ListCapacityTasksstart-capacity-task
/ StartCapacityTask
In addition to the same options available within the console, it is possible to request a dry run of the capacity task to determine if the instance type and instance size changes are above or below the available instance capacity. Requesting a dry run doesn’t make any changes to your plan.
For example, using the CLI to submit a capacity task to homogeneously slot an Outpost with 2 x m5 and 2 x c5 hosts (192 vCPU for each capacity pool) with xlarge instance sizes, and using periodic running could be achieved by running the following:
aws outposts start-capacity-task \
--outpost-identifier outpost-arn \
--instance-pools '[{"InstanceType":"c5.xlarge","Count":48},{"InstanceType":"m5.xlarge","Count":48}]' \
--task-action-on-blocking-instances WAIT_FOR_EVACUATION
Conclusion
This post demonstrated how to run a capacity task on Outposts and view your existing capacity configuration. For more information on how to manage and monitor your capacity configuration on Outposts, see Capacity management for AWS Outposts user guide and the Capacity planning section of the AWS Outposts High Availability Design and Architecture Considerations whitepaper, and the Modify AWS Outposts instance capacity – Outposts rack/Modify AWS Outposts instance capacity – Outposts server user guide sections for your respective environment. Reach out to your AWS account team to learn more about Outposts and self-service capacity management.