How do I stop my Amazon EC2 Windows instance from signaling back as CREATE_COMPLETE before it finishes bootstrapping?

Last updated: 2019-04-24

I'm bootstrapping an Amazon Elastic Compute Cloud (Amazon EC2) Windows instance using the cfn-init (cfn-init.exe) and cfn-signal (cfn-signal.exe) helper scripts in AWS CloudFormation. My cfn-init script signals back too early, and AWS CloudFormation marks my Windows instance as CREATE_COMPLETE before it finishes bootstrapping. How can I fix this?

Short Description

In a Windows instance, UserData scripts are executed by the Ec2ConfigService process. UserData invokes cfn-init.exe, which runs as a child process of Ec2ConfigService.

Your Windows instance could be signaling back as CREATE_COMPLETE before it finishes bootstrapping for the following reasons:

  • If one of the steps executed by cfn-init.exe requires a system reboot, the system can shut down, and then hand execution back to the Ec2ConfigService process. The system continues processing the UserData script, and then executes cfn-signal.exe, which signals back to AWS CloudFormation.
  • The cfn-signal doesn't signal back after a reboot, because UserData runs only once.

In the code example below, cfn-signal.exe is invoked directly from UserData. If the cfn-init.exe process performs a reboot, then the cfn-signal.exe command can't be invoked, because UserData runs only once:

"UserData": { "Fn::Base64": { "Fn::Join": ["", [
    "\",
    " cfn-init.exe -v -s ", { "Ref": "AWS::StackName" },
    "  -r Instance",
    "  --region ", { "Ref" : "AWS::Region" }, "\",
    " cfn-signal.exe -e %ERRORLEVEL% ", {"Fn::Base64": { "Ref": "WaitHandle" }}, "\",
    ""
]]}}

Resolution

To stop your Windows instance from signaling back as CREATE_COMPLETE before it finishes bootstrapping, refer to the sample AWS CloudFormation template below, and then complete the following steps:

1.    Use configsets in the cfn-init metadata to separate the configurations that require a reboot from the configurations that don't require a reboot.

2.    Move the cfn-signal.exe from the UserData section of the AWS::EC2::Instance or AWS::AutoScaling::LaunchConfiguration resource to the AWS::CloudFormation::Init metadata section of the template.

3.    Execute cfn-signal.exe as the last command run by the last configset.

4.    Modify UserData to cfn-init and specify the ascending configset.

Example AWS CloudFormation template:

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "cfn-init example using configsets",
  "Parameters": {
    "AMI": {
      "Type": "AWS::EC2::Image::Id"
    }
  },
  "Resources": {
    "WindowsInstance": {
      "Metadata": {
        "AWS::CloudFormation::Init": {
          "configSets": {
            "ascending": [
              "config1",
              "config2",
              "config3"
            ]
          },
          "config1": {
            "files": {
              "C:\\setup\\setenvironment.ps1": {
                "content": {
                  "Fn::Join": [
                    "",
                    [
                      "$folder = 'C:\\Program Files\\Server\\packages\\bin.20182.18.0826.0815\\'\n",
                      "$OldPath = [System.Environment]::GetEnvironmentVariable('path')\n",
                      "$NewPath = $OldPath+ ';' + $Folder\n",
                      "[System.Environment]::SetEnvironmentVariable('path',$NewPath,'Machine')"
                    ]
                  ]
                }
              }
            }
          },
          "config2": {
            "commands": {
              "0-restart": {
                "command": "powershell.exe -Command Restart-Computer",
                "waitAfterCompletion": "forever"
              },
              "1-signal-success": {
                "command": {
                  "Fn::Join": [
                    "",
                    [
                      "cfn-signal.exe -e 0 --stack ",
                      {
                        "Ref": "AWS::StackId"
                      },
                      " --resource WindowsInstance --region ",
                      {
                        "Ref": "AWS::Region"
                      }
                    ]
                  ]
                }
              }
            }
          },
          "config3": {
            "commands": {
              "01-setenvironment": {
                "command": "powershell.exe -ExecutionPolicy Unrestricted c:\\setup\\setenvironment.ps1"
              },
              "waitAfterCompletion": "0"
            },
            "02-signal-resource": {
              "command": {
                "Fn::Join": [
                  "",
                  [
                    "cfn-signal.exe -e %ERRORLEVEL% --resource WindowsServer --stack ",
                    {
                      "Ref": "AWS::StackName"
                    }
                  ]
                ]
              }
            }
          }
        }
      },
      "Type": "AWS::EC2::Instance",
      "Properties": {
        "ImageId": {
          "Ref": "AMI"
        },
        "InstanceType": "t2.medium",
        "UserData": {
          "Fn::Base64": {
            "Fn::Join": [
              "",
              [
                "<script>\n",
                "cfn-init.exe -v -s ",
                {
                  "Ref": "AWS::StackId"
                },
                " -r WindowsInstance",
                " --configsets ascending",
                " --region ",
                {
                  "Ref": "AWS::Region"
                },
                "\n- </script>\n"
              ]
            ]
          }
        }
      },
      "CreationPolicy": {
        "ResourceSignal": {
          "Count": "1",
          "Timeout": "PT30M"
        }
      }
    }
  }
}

The signal is no longer running in UserData, which means that you can't retrieve the exit code provided by the cfn-init process. The default behavior is fail create/update if a signal isn't received. When this happens, then the stack returns a "timeout exceeded" error. To troubleshoot any failures, use the logs at c:\cfn\log in the Windows instance.

5.    Set the waitAfterCompletion parameter to forever.

Note: The default value of waitAfterCompletion is 60 seconds. If you change the value to forever, cfn-init will exit and then resume only after the reboot is complete.


Did this article help you?

Anything we could improve?


Need more help?