How do I stop my Amazon EC2 Windows instance from signaling back as CREATE_COMPLETE before it finishes bootstrapping?

Last updated: 2019-04-24

I'm bootstrapping an Amazon Elastic Compute Cloud (Amazon EC2) Windows instance using the cfn-init (cfn-init.exe) and cfn-signal (cfn-signal.exe) helper scripts in AWS CloudFormation. My cfn-init script signals back too early, and AWS CloudFormation marks my Windows instance as CREATE_COMPLETE before it finishes bootstrapping. How can I fix this?

Short Description

In a Windows instance, UserData scripts are executed by the Ec2ConfigService process. UserData invokes cfn-init.exe, which runs as a child process of Ec2ConfigService.

Your Windows instance could be signaling back as CREATE_COMPLETE before it finishes bootstrapping for the following reasons:

  • If one of the steps executed by cfn-init.exe requires a system reboot, the system can shut down, and then hand execution back to the Ec2ConfigService process. The system continues processing the UserData script, and then executes cfn-signal.exe, which signals back to AWS CloudFormation.
  • The cfn-signal doesn't signal back after a reboot, because UserData runs only once.

In the code examples below, cfn-signal.exe is invoked directly from UserData. If the cfn-init.exe process performs a reboot, then the cfn-signal.exe command can't be invoked, because UserData runs only once.

JSON example:

"UserData": {
  "Fn::Base64": {
	"Fn::Join": [
	  "",
	  [
		"<script>\n",
		"cfn-init.exe -v -s ",
		{
		  "Ref": "AWS::StackId"
		},
		" -r WindowsInstance",
		" --configsets ascending",
		" --region ",
		{
		  "Ref": "AWS::Region"
		},
		"\n",
		"cfn-signal.exe -e %ERRORLEVEL% --stack ",
		{
		  "Ref": "AWS::StackId"
		},
		" --resource WindowsInstance --region ",
		{
		  "Ref": "AWS::Region"
		},
		"\n",
		"</script>"
	  ]
	]
  }
}

YAML example:

UserData:
Fn::Base64:
  Fn::Join:
  - ''
  - - "<script>\n"
	- 'cfn-init.exe -v -s '
	- Ref: AWS::StackId
	- " -r WindowsInstance"
	- " --configsets ascending"
	- " --region "
	- Ref: AWS::Region
	- "\n"
	- 'cfn-signal.exe -e %ERRORLEVEL% --stack '
	- Ref: AWS::StackId
	- " --resource WindowsInstance --region "
	- Ref: AWS::Region
	- "\n"
	- "</script>"

Resolution

To stop your Windows instance from signaling back as CREATE_COMPLETE before it finishes bootstrapping, refer to the sample AWS CloudFormation templates below, and then complete the following steps:

1.    Use configsets in the cfn-init Metadata section of your template to separate the configurations that require a reboot from the configurations that don't require a reboot.

2.    Move the cfn-signal.exe from the UserData section of the AWS::EC2::Instance or AWS::AutoScaling::LaunchConfiguration resource to the AWS::CloudFormation::Init Metadata section of the template.

3.    Execute cfn-signal.exe as the last command run by the last configset.

4.    In your JSON or YAML template, change UserData to cfn-init and specify the ascending configset.

JSON example:

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "cfn-init example using configsets",
  "Parameters": {
    "AMI": {
      "Type": "AWS::EC2::Image::Id"
    }
  },
  "Resources": {
    "WindowsInstance": {
      "Metadata": {
        "AWS::CloudFormation::Init": {
          "configSets": {
            "ascending": [
              "config1",
              "config2",
              "config3"
            ]
          },
          "config1": {
            "files": {
              "C:\\setup\\setenvironment.ps1": {
                "content": {
                  "Fn::Join": [
                    "",
                    [
                      "$folder = 'C:\\Program Files\\Server\\packages\\bin.20182.18.0826.0815\\'\n",
                      "$OldPath = [System.Environment]::GetEnvironmentVariable('path')\n",
                      "$NewPath = $OldPath+ ';' + $Folder\n",
                      "[System.Environment]::SetEnvironmentVariable('path',$NewPath,'Machine')"
                    ]
                  ]
                }
              }
            }
          },
          "config2": {
            "commands": {
              "0-restart": {
                "command": "powershell.exe -Command Restart-Computer",
                "waitAfterCompletion": "forever"
              }
            }
          },
          "config3": {
            "commands": {
              "01-setenvironment": {
                "command": "powershell.exe -ExecutionPolicy Unrestricted c:\\setup\\setenvironment.ps1"
              },
              "waitAfterCompletion": "0"
            },
            "02-signal-resource": {
              "command": {
                "Fn::Join": [
                  "",
                  [
                    "cfn-signal.exe -e %ERRORLEVEL% --resource WindowsInstance --stack ",
                    {
                      "Ref": "AWS::StackName"
                    }
                  ]
                ]
              }
            }
          }
        }
      },
      "Type": "AWS::EC2::Instance",
      "Properties": {
        "ImageId": {
          "Ref": "AMI"
        },
        "InstanceType": "t2.medium",
        "UserData": {
          "Fn::Base64": {
            "Fn::Join": [
              "",
              [
                "<script>\n",
                "cfn-init.exe -v -s ",
                {
                  "Ref": "AWS::StackId"
                },
                " -r WindowsInstance",
                " --configsets ascending",
                " --region ",
                {
                  "Ref": "AWS::Region"
                },
                "\n</script>"
              ]
            ]
          }
        }
      },
      "CreationPolicy": {
        "ResourceSignal": {
          "Count": "1",
          "Timeout": "PT30M"
        }
      }
    }
  }
}

YAML example:

AWSTemplateFormatVersion: '2010-09-09'
Description: cfn-init example using configsets
Parameters:
  AMI:
    Type: AWS::EC2::Image::Id
Resources:
  WindowsInstance:
    Metadata:
      AWS::CloudFormation::Init:
        configSets:
          ascending:
          - config1
          - config2
          - config3
        config1:
          files:
            C:\setup\setenvironment.ps1:
              content:
                Fn::Join:
                - ''
                - - "$folder = 'C:\\Program Files\\Server\\packages\\bin.20182.18.0826.0815\\'\n"
                  - "$OldPath = [System.Environment]::GetEnvironmentVariable('path')\n"
                  - "$NewPath = $OldPath+ ';' + $Folder\n"
                  - "[System.Environment]::SetEnvironmentVariable('path',$NewPath,'Machine')"
        config2:
          commands:
            0-restart:
              command: powershell.exe -Command Restart-Computer
              waitAfterCompletion: forever
        config3:
          commands:
            01-setenvironment:
              command: powershell.exe -ExecutionPolicy Unrestricted c:\setup\setenvironment.ps1
            waitAfterCompletion: '0'
          02-signal-resource:
            command:
              Fn::Join:
              - ''
              - - 'cfn-signal.exe -e %ERRORLEVEL% --resource WindowsInstance --stack '
                - Ref: AWS::StackName
    Type: AWS::EC2::Instance
    Properties:
      ImageId:
        Ref: AMI
      InstanceType: t2.medium
      UserData:
        Fn::Base64:
          Fn::Join:
          - ''
          - - "<script>\n"
            - 'cfn-init.exe -v -s '
            - Ref: AWS::StackId
            - " -r WindowsInstance"
            - " --configsets ascending"
            - " --region "
            - Ref: AWS::Region
            - "\n</script>"
    CreationPolicy:
      ResourceSignal:
        Count: '1'
        Timeout: PT30M

In these templates, the signal is no longer running in UserData, which means that you can't retrieve the exit code provided by the cfn-init process. By default, AWS CloudFormation fails to create or update the stack if a signal isn't received from UserData or Metadata. Then, the stack returns a "timeout exceeded" error. To troubleshoot any failures, use the logs at c:\cfn\log in the Windows instance.

5.    Set the waitAfterCompletion parameter to forever.

Note: The default value of waitAfterCompletion is 60 seconds. If you change the value to forever, cfn-init exits and then resumes only after the reboot is complete.


Did this article help you?

Anything we could improve?


Need more help?