Seamlessly off-board from AWS OpsWorks Stacks by detaching resources

Today, we are announcing new capabilities that enable customers to off-board their stacks, layers, and instances from AWS OpsWorks Stacks before it reaches End of Life on May 26, 2024.

Previously, we released this blog post that provides customers a way to migrate workloads in a blue/green manner from OpsWorks Stacks to a new architecture that leverages AWS CloudFormation, AWS Systems Manager, and Amazon EC2 Auto Scaling groups. While this is still a recommended approach, we have received feedback from some of our customer base that they would like to retain their existing OpsWorks instances and manage them freely outside of the OpsWorks Stacks service.

To address this feedback, we are releasing new capabilities that enables you to detach existing stacks, layers, and instances from the OpsWorks Stacks service. It freely manages them on Amazon Elastic Compute Cloud (Amazon EC2), AWS Systems Manager, or whatever Amazon EC2 compatible tooling you see fit. These new capabilities allow customers the freedom to choose how they configure, manage and orchestrate the AWS resources that OpsWorks originally provisioned and managed on their behalf.

In this blog post, we will explore how to use this new tool to detach resources from OpsWorks Stacks.

Detach from OpsWorks Stacks

The detach process disassociates your OpsWorks instances from the OpsWorks Stacks service. While the instances will remain in your AWS account after the detachment, you will no longer be able to manage them using the OpsWorks service. Instead, you will be able to use Amazon EC2, AWS Systems Manager, or any EC2 compatible approach to configure and manage them.

At a high level, the process involves the following steps:

1. The tool performs validation checks to ensure resources are ready for migration.
2. It exports the Custom JSON from your OpsWorks Stack and store it as an object in Amazon Simple Storage Service (Amazon S3).
3. It creates System Manager Automation documents representing each OpsWorks Stacks lifecycle event.
4. It creates AWS Service Catalog AppRegistry for all instances that are due to be detached and detach any Elastic Load Balancing (ELB) Load Balancers from the OpsWorks layers.
5. Finally, it detaches and deregisters other resources including Amazon Relational Database Service (Amazon RDS) instances.

Follow these steps to use the tool:

1. Install the script.
2. Detach your layers from OpsWorks Stacks.
3. Operate your resources post migration.

Prerequisites

For this walkthrough, you can use AWS CloudShell or ensure that your local machine or Amazon EC2 instance has the following prerequisites set up:

• Python
• AWS CLI

Known Limitations

• You need to disable load-based or time-based scaling for any OpsWorks instances you are trying to detach prior to running the script.
• The tooling doesn’t create or generate IAM entities specified in the Permission page of the OpsWorks console.
• The tooling doesn’t export any information related to the apps associated with your OpsWorks stacks. Apps are not supported or migrated.
• If you rely on internal OpsWorks recipes or data bags, detaching or deregistering the instance will remove those dependencies and might leave your instance in an undesired state. You should update your cookbooks to stop relying on these before attempting to detach.

Install the Script

The script is written in Python and requires installation of a number of packages in order to function. In this step, you will download and install the script in your local machine, Amazon EC2 instance in your account, or by using AWS CloudShell.

1. Download the zip file that contains the detach script and all the relevant files by running the below command:

aws s3api get-object \
 --bucket detach-in-place-bucket-prod-us-east-1 \
 --key detach_in_place_script.zip detach_in_place_script.zip

2. Unzip the file:

unzip detach_in_place_script.zip

3. Install pipenv if necessary:

pip install pipenv

4. Run the following commands to install all required python packages:

pipenv install -r requirements.txt 
pipenv shell

5. You can look at the available options when utilizing the ‘detach’, ‘handle-prerequisites’ and ‘cleanup’ commands as follows:

• • python3 layer_detacher.py detach –help
  • python3 layer_detacher.py handle-prerequisites –help
  • python3 layer_detacher.py cleanup –help

Detach your layers from OpsWorks Stacks

In this step, you will detach the resources in your layer from OpsWorks.

Note: You may utilize these commands independently of one another, for example, you are not required to run ‘handle-prerequisites’ prior to ‘detach’.

1. Check whether the OpsWorks instances in the layer meet the criteria for migration. You can find more about the specific criteria in using the AWS OpsWorks stacks detach in place tool.

python3 layer_detacher.py handle-prerequisites \ 
 --layer-id <OPSWORKS_LAYER_ID> \
 --region <REGION>

2. Once the pre-requisites have been met, run the below command to detach the instance from the layer.
   
   Note: This command iterates overall instances in the layer and attempts to detach them in parallel. If you would prefer to detach Instances in batches instead, you can use the batch-size parameter. For details on how to do this, please refer to using the AWS OpsWorks stacks detach in place tool.

python3 layer_detacher.py detach \
 --layer-id <OPSWORKS_LAYER_ID> \
 --region <REGION>

3. Run the below command to clean up the OpsWorks Layer and delete it. Once the layer is deleted, you can then do the same for the OpsWorks Stack. The command iterates over all resources within a layer and delete them.
   
   Note: We recommend to run the new setup and validate for a few days prior to executing the cleanup command. This ensures that any necessary configurations from the stack are readily available if needed.

python3 layer_detacher.py cleanup \
  --layer-id <OPSWORKS_LAYER_ID> \
  --region <REGION>

Once you have cleaned up a given layer, you may want to perform the same action on multiple layers within a stack.

4. Finally, run the below command to clean up and delete the OpsWorks Stack.
   
   Note: We recommend to run the new setup and validate for a few days prior to executing the cleanup command.

python3 layer_detacher.py cleanup \
  --stack-id <OPSWORKS_STACK_ID> \
  --region <REGION>

Operate your resources post migration

After executing the detach command, the tool creates a new Service Catalog AppRegistry application corresponding to the detached layer. The application name follows the format <Layer_name>---<Layer_ID>. It also adds the OpsWorksLayerId tag to uniquely identify the application matching the detached layer.

To add new AWS resources to this application, you can do one of the following:

1. Tag the resource with the unique application tag of the AppRegistry application:
   Tag Key: awsApplication
   Value: arn:aws:resource-groups:<REGION>:<ACCOUNT_ID>:group/<APPLICATION_NAME>/<APPLICATION_ID>
2. Additionally, for each AppRegistry application, it creates AWS Resource Groups with the following tags.

• • • EnableAWSServiceCatalogAppRegistry
    • aws:servicecatalog:applicationName
    • aws:servicecatalog:applicationId
    • aws:servicecatalog:applicationArn

You can perform the below tasks after detachment:

Conclusion

In this blog, we have outlined a method for off-boarding resources from OpsWorks using the provided script. This script facilitates transitioning from OpsWorks stacks to Systems Manager without the need for blue/green setup or replacement infrastructures.

For further details, refer to the following resources:

Migrate your AWS OpsWorks stacks to AWS Systems Manager
Execute chef recipes using AWS Systems Manager
EOL FAQs

About the authors: