How do I stop my Amazon EC2 Windows instance from signaling back as CREATE_COMPLETE before the instance finishes bootstrapping?

Last updated: 2019-11-15

I'm bootstrapping an Amazon Elastic Compute Cloud (Amazon EC2) Windows instance using the cfn-init (cfn-init.exe) and cfn-signal (cfn-signal.exe) helper scripts in AWS CloudFormation. My cfn-init script signals back too early. Then, AWS CloudFormation marks my Windows instance as CREATE_COMPLETE before the instance finishes bootstrapping. How can I fix this?

Short Description

In a Windows instance, UserData scripts are executed by the Ec2ConfigService process. UserData invokes cfn-init.exe, which runs as a child process of Ec2ConfigService.

Your Windows instance could be signaling back as CREATE_COMPLETE for the following reasons:

  • If one of the steps executed by cfn-init.exe requires a system reboot, the system can shut down, and then return execution back to the Ec2ConfigService process. The system continues processing the UserData script, and then executes cfn-signal.exe, which signals back to AWS CloudFormation.
  • The cfn-signal doesn't signal back after a reboot, because UserData runs only once.

In the code examples below, cfn-signal.exe is invoked directly from UserData. If the cfn-init.exe process performs a reboot, then the cfn-signal.exe command can't be invoked, because UserData runs only once.

JSON example:

"UserData": {
  "Fn::Base64": {
    "Fn::Join": [
      "",
      [
        "<script>\n",
        "cfn-init.exe -v -s ",
        {
          "Ref": "AWS::StackId"
        },
        " -r WindowsInstance",
        " --configsets ascending",
        " --region ",
        {
          "Ref": "AWS::Region"
        },
        "\n",
        "cfn-signal.exe -e %ERRORLEVEL% --stack ",
        {
          "Ref": "AWS::StackId"
        },
        " --resource WindowsInstance --region ",
        {
          "Ref": "AWS::Region"
        },
        "\n",
        "</script>"
      ]
    ]
  }
}

YAML example:

UserData:
Fn::Base64:
  Fn::Join:
  - ''
  - - "<script>\n"
    - 'cfn-init.exe -v -s '
    - Ref: AWS::StackId
    - " -r WindowsInstance"
    - " --configsets ascending"
    - " --region "
    - Ref: AWS::Region
    - "\n"
    - 'cfn-signal.exe -e %ERRORLEVEL% --stack '
    - Ref: AWS::StackId
    - " --resource WindowsInstance --region "
    - Ref: AWS::Region
    - "\n"
    - "</script>"

Resolution

1.    Use configsets in the cfn-init Metadata section of your template to separate the configurations that require a reboot from the configurations that don't require a reboot.

2.    Move the cfn-signal.exe from the UserData section of the AWS::EC2::Instance or AWS::AutoScaling::LaunchConfiguration resource to the AWS::CloudFormation::Init Metadata section of the template.

3.    Execute cfn-signal.exe as the last command run by the last configset.

4.    In your JSON or YAML template, change UserData to cfn-init and specify the ascending configset.

JSON example:

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Description": "cfn-init example using configsets",
    "Parameters": {
        "AMI": {
            "Type": "AWS::EC2::Image::Id"
        }
    },
    "Resources": {
        "WindowsInstance": {
            "Metadata": {
                "AWS::CloudFormation::Init": {
                    "configSets": {
                        "ascending": [
                            "config1",
                            "config2",
                            "config3"
                        ]
                    },
                    "config1": {
                        "files": {
                            "C:\\setup\\setenvironment.ps1": {
                                "content": {
                                    "Fn::Join": [
                                        "",
                                        [
                                            "$folder = 'C:\\Program Files\\Server\\packages\\bin.20182.18.0826.0815\\'\n",
                                            "$OldPath = [System.Environment]::GetEnvironmentVariable('path')\n",
                                            "$NewPath = $OldPath+ ';' + $Folder\n",
                                            "[System.Environment]::SetEnvironmentVariable('path',$NewPath,'Machine')"
                                        ]
                                    ]
                                }
                            }
                        }
                    },
                    "config2": {
                        "commands": {
                            "0-restart": {
                                "command": "powershell.exe -Command Restart-Computer",
                                "waitAfterCompletion": "forever"
                            }
                        }
                    },
                    "config3": {
                        "commands": {
                            "01-setenvironment": {
                                "command": "powershell.exe -ExecutionPolicy Unrestricted c:\\setup\\setenvironment.ps1"
                            },
                            "waitAfterCompletion": "0"
                        },
                        "02-signal-resource": {
                            "command": {
                                "Fn::Join": [
                                    "",
                                    [
                                        "cfn-signal.exe -e %ERRORLEVEL% --resource WindowsInstance --stack ",
                                        {
                                            "Ref": "AWS::StackName"
                                        },
                                        "         --region ",
                                        {
                                            "Ref": "AWS::Region"
                                        }
                                    ]
                                ]
                            }
                        }
                    }
                }
            },
            "Type": "AWS::EC2::Instance",
            "Properties": {
                "ImageId": {
                    "Ref": "AMI"
                },
                "InstanceType": "t2.medium",
                "UserData": {
                    "Fn::Base64": {
                        "Fn::Join": [
                            "",
                            [
                                "<script>\n",
                                "cfn-init.exe -v -s ",
                                {
                                    "Ref": "AWS::StackId"
                                },
                                " -r WindowsInstance",
                                " --configsets ascending",
                                " --region ",
                                {
                                    "Ref": "AWS::Region"
                                },
                                "\n</script>"
                            ]
                        ]
                    }
                }
            },
            "CreationPolicy": {
                "ResourceSignal": {
                    "Count": "1",
                    "Timeout": "PT30M"
                }
            }
        }
    }
}

YAML example:

AWSTemplateFormatVersion: 2010-09-09
Description: cfn-init example using configsets
Parameters:
  AMI:
    Type: 'AWS::EC2::Image::Id'
Resources:
  WindowsInstance:
    Metadata:
      'AWS::CloudFormation::Init':
        configSets:
          ascending:
            - config1
            - config2
            - config3
        config1:
          files:
            'C:\setup\setenvironment.ps1':
              content: !Join 
                - ''
                - - >
                    $folder = 'C:\Program
                    Files\Server\packages\bin.20182.18.0826.0815\'
                  - >
                    $OldPath =
                    [System.Environment]::GetEnvironmentVariable('path')
                  - |
                    $NewPath = $OldPath+ ';' + $Folder
                  - >-
                    [System.Environment]::SetEnvironmentVariable('path',$NewPath,'Machine')
        config2:
          commands:
            0-restart:
              command: powershell.exe -Command Restart-Computer
              waitAfterCompletion: forever
        config3:
          commands:
            01-setenvironment:
              command: >-
                powershell.exe -ExecutionPolicy Unrestricted
                c:\setup\setenvironment.ps1
            waitAfterCompletion: '0'
          02-signal-resource:
            command: !Join 
              - ''
              - - >-
                  cfn-signal.exe -e %ERRORLEVEL% --resource WindowsInstance
                  --stack 
                - !Ref 'AWS::StackName'
                - '         --region '
                - !Ref 'AWS::Region'
    Type: 'AWS::EC2::Instance'
    Properties:
      ImageId: !Ref AMI
      InstanceType: t2.medium
      UserData: !Base64 
        'Fn::Join':
          - ''
          - - |
              <script>
            - 'cfn-init.exe -v -s '
            - !Ref 'AWS::StackId'
            - ' -r WindowsInstance'
            - ' --configsets ascending'
            - ' --region '
            - !Ref 'AWS::Region'
            - |-
              </script>
    CreationPolicy:
      ResourceSignal:
        Count: '1'
        Timeout: PT30M

In the preceding templates, the signal is no longer running in UserData, which means that you can't retrieve the exit code provided by the cfn-init process. By default, AWS CloudFormation fails to create or update the stack if a signal isn't received from UserData or Metadata. Then, the stack returns a "timeout exceeded" error. To troubleshoot any failures, use the logs at c:\cfn\log in the Windows instance.

5.    Set the waitAfterCompletion parameter to forever.

Note: The default value of waitAfterCompletion is 60 seconds. If you change the value to forever, cfn-init exits and then resumes only after the reboot is complete.


Did this article help you?

Anything we could improve?


Need more help?