How can I stop an instance in "stop_failed" status in AWS OpsWorks when I receive an "Internal Error" message?

Last updated: 2020-11-02

I can't stop instances marked as "stop_failed" in AWS OpsWorks Stacks. My AWS Identity and Access Management (IAM) service role has all the required permissions, but when I try to stop my instances through the AWS Management Console, I receive an "Internal Error" message. Sometimes, I see there's no Amazon Elastic Compute Cloud (Amazon EC2) instance associated with my OpsWorks Stacks instances. How can I safely mark my OpsWorks Stacks instances as stopped?

Short description

You can't stop an instance in OpsWorks Stacks if the instance was deleted in Amazon EC2. To find the instance that needs to be stopped, the OpsWorks StopInstance API call requires an EC2 instance ID and related permissions in the IAM service role. When OpsWorks Stacks can't locate the EC2 ID, the API call fails and returns an "Internal Error" message. For StopInstance to work when the underlying EC2 instance was already deleted or when there's an internal error, you must use the --force clause.

Resolution

Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.

Get the OpsWorks instance IDs of the instances that need to be marked as stopped

1.    To filter the instances in the stopped_failed state for a given stack, run the describe-instances command with a query:

$ aws opsworks describe-instances --stack-id your-stack-id --region your-region --query 'Instances[?Status=='stop_failed'].{Id:InstanceId}'

2.    Note the instance IDs that you want to mark as stopped.

Note: If you're using the classic OpsWorks Stacks endpoint, then use us-east-1 as the AWS Region.

Use the --force clause to stop the instances

If you have a single instance that you want to mark as stopped, then run the stop-instance command:

$ aws opsworks stop-instance --instance-id your-opsworks-instance-id --force --region your-region

If you have several instances, then you can iterate through the instances using custom logic. For example: 

$ for Id in $(aws opsworks describe-instances --stack-id your-stack-id --region your-region --query 'Instances[?Status==`start_failed`].{Id:InstanceId}' --output text) ; do aws opsworks stop-instance --instance-id $Id --force --region your-region  ; done

Did this article help?


Do you need billing or technical support?