AWS Compute Blog

Optimizing Joining Windows Server Instances to a Domain with PowerShell in AWS CloudFormation


Scott Zimmerman
AWS Solutions Architect

Deploying applications with Active Directory, including Microsoft SharePoint Server and custom .NET applications, can take several minutes and possibly even hours. My colleague Julien Lépine wrote an excellent post, Optimize AWS CloudFormation Templates, about parallelizing Amazon EC2 instance creation in AWS CloudFormation. His post shows a general technique for using cfn-signal after an instance completes its initialization to trigger other dependent instances (which were launched in parallel) to begin initialization. Launching multiple instances simultaneously, and then blocking initialization only when necessary, can significantly reduce total stack creation time.

This post describes a specialization of his technique as it applies to connecting Microsoft Windows Server instances to a domain. Often, a domain controller needs to be set up before an application server can connect to the domain. To reduce stack creation time, you can launch those two servers in parallel, as long as there is a mechanism to block the application server from attempting to join the domain until the domain controller is fully initialized. Using the DependsOn element in the application server instance is one solution. A faster technique is to use cfn-signal. And it’s even faster to create the domain controller as a custom AMI.

The following diagram shows an example infrastructure with these four components:

  • NAT gateway in the public subnet for allowing outbound Internet traffic from the private subnet
  • BastionHost1 in the public subnet for allowing inbound administrative remote access into the private subnet
  • Server1 in the private subnet to represent a typical application server that needs to join the Active Directory domain
  • DC1 , where the Active Directory role is installed

Example infrastructure for an application server and domain controller

I’m deliberately keeping this simple with a single Availability Zone so we can focus on the concept of domain joining. Of course, in a production environment, you would have two Availability Zones and redundant instances for high availability.

There are many variables that affect total stack creation time, including instance types, regions, and other intangibles. I timed three approaches (described later) to building the infrastructure shown in the diagram and got the following results using m4.large for the domain controllers (your mileage will vary).

Template Method Timing
Domain-Join1.json Standard AMI with CreationPolicy and DependsOn About 20 minutes
Domain-Join2.json Standard AMI with cfn-signal and a PowerShell loop About 16 minutes
Domain-Join3.json Custom AMI with the AD role pre-installed About 7 minutes

The CloudFormation templates are included below so you can try it yourself. You just need to replace the AMIID parameter with the latest AMI ID for Windows Server 2012 R2 Base in your preferred region, and provide a key pair for your region.

After you deploy the stack in CloudFormation, you can remotely connect to the BastionHost1 instance using the administrator password you specified when you launched the stack. Then from the BastionHost1 instance, you can run Remote Desktop Connection to the private IP address of either Server1 or DC1. Note that you log in as the local administrator to BastionHost1 and Server1, but use mydomain\StackAdmin with your selected password when logging into DC1.

Now let’s dive into each of these techniques.

Using the Windows AMI with CreationPolicy and DependsOn

Filename: Domain-Join1.json

{
  "Description": "CloudFormation template for domain join with CreationPolicy and DependsOn", 
  "Parameters": {
    "KeyName": {
      "MinLength" : 1,  
      "Type": "AWS::EC2::KeyPair::KeyName"
    },
    "BaseAmiId": {
      "Default": "ami-bd3ba0aa",
      "Type": "String"
    },
    "DomainAdminUser": {
      "Description": "User name for the account that will be added as Domain Administrator. This is separate from the default \"Administrator\" account",
      "Type": "String",
      "Default": "StackAdmin"
    },
    "AdminPassword": {
        "NoEcho": "true",
        "Description" : "The Windows administrator account password",
        "Type": "String",
        "MinLength": "8",
        "MaxLength": "41"
    },
    "DomainDNSName": {
      "Description": "Fully qualified domain name (FQDN) of the forest root domain",
      "Type": "String",
      "Default": "mydomain.local"
    },
    "DomainNetBiosName": {
      "Description": "Netbios name for the domain",
      "Type": "String",
      "Default": "mydomain"
    }
  },
  "Resources": {
    "BastionHost1": {
      "Properties": {
        "ImageId": {
          "Ref": "BaseAmiId"
        },
        "InstanceType": "t2.large",
        "KeyName": {
          "Ref": "KeyName"
        },
        "NetworkInterfaces": [
          {
            "AssociatePublicIpAddress": "true",
            "DeleteOnTermination": "true",
            "DeviceIndex": "0",
            "GroupSet": [
              {
                "Ref": "BastionSecurityGroup"
              }
            ],
            "PrivateIpAddress": "10.1.1.100",
            "SubnetId": {
              "Ref": "PublicSubnetAZ1"
            }
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "BastionHost1"
          }
        ],
        "UserData": {
          "Fn::Base64": {
            "Fn::Join": [
              "",
              [
                "<powershell>\n",
                "### Change local admin password...\n",
                "([adsi]\"WinNT://$env:computername/Administrator\").SetPassword('",
                {
                  "Ref": "AdminPassword"
                },
                "') \n",
                "</powershell>\n"       
              ]
            ]
          }
        }
      },
      "Type": "AWS::EC2::Instance"
    },  
    "DC1": {    
      "Type": "AWS::EC2::Instance",
      "DependsOn": "NATRoute",
      "CreationPolicy" : {
        "ResourceSignal" : {
          "Timeout": "PT20M",
          "Count"  : "1"
        }
      },      
      "Metadata": {
        "AWS::CloudFormation::Init": {
          "configSets": {
            "config": [
              "setup",
              "rename",
              "installADDS"
            ]
          },
          "setup": {
            "files": {
              "c:\\cfn\\cfn-hup.conf": {
                "content": {
                  "Fn::Join": [
                    "",
                    [
                      "[main]\n",
                      "stack=",
                      {
                        "Ref": "AWS::StackName"
                      },
                      "\n",
                      "region=",
                      {
                        "Ref": "AWS::Region"
                      },
                      "\n"
                    ]
                  ]
                }
              },
              "c:\\cfn\\hooks.d\\cfn-auto-reloader.conf": {
                "content": {
                  "Fn::Join": [
                    "",
                    [
                      "[cfn-auto-reloader-hook]\n",
                      "triggers=post.update\n",
                      "path=Resources.DC1.Metadata.AWS::CloudFormation::Init\n",
                      "action=cfn-init.exe -v -c config -s ",
                      {
                        "Ref": "AWS::StackId"
                      },
                      " -r DC1",
                      " --region ",
                      {
                        "Ref": "AWS::Region"
                      },
                      "\n"
                    ]
                  ]
                }
              },
              "c:\\cfn\\scripts\\Set-StaticIP.ps1": {
                "content": {
                  "Fn::Join": [
                    "",
                    [
                      "$netip = Get-NetIPConfiguration;",
                      "$ipconfig = Get-NetIPAddress | ?{$_.IpAddress -eq $netip.IPv4Address.IpAddress};",
                      "Get-NetAdapter | Set-NetIPInterface -DHCP Disabled;",
                      "Get-NetAdapter | New-NetIPAddress -AddressFamily IPv4 -IPAddress $netip.IPv4Address.IpAddress -PrefixLength $ipconfig.PrefixLength -DefaultGateway $netip.IPv4DefaultGateway.NextHop;",
                      "Get-NetAdapter | Set-DnsClientServerAddress -ServerAddresses $netip.DNSServer.ServerAddresses;",
                      "\n"
                    ]
                  ]
                }
              },
              "c:\\cfn\\scripts\\ConvertTo-EnterpriseAdmin.ps1": {
                "source": "https://s3.amazonaws.com/quickstart-reference/microsoft/activedirectory/latest/scripts/ConvertTo-EnterpriseAdmin.ps1"
              }
            },
            "services": {
              "windows": {
                "cfn-hup": {
                  "enabled": "true",
                  "ensureRunning": "true",
                  "files": [
                    "c:\\cfn\\cfn-hup.conf",
                    "c:\\cfn\\hooks.d\\cfn-auto-reloader.conf"
                  ]
                }
              }
            },
            "commands": {
              "a-disable-win-fw": {
                "command": {
                  "Fn::Join": [
                    "",
                    [
                      "powershell.exe -Command \"Get-NetFirewallProfile | Set-NetFirewallProfile -Enabled False"
                    ]
                  ]
                },
                "waitAfterCompletion": "0"
              }
            }
          },
          "rename": {
            "commands": {
              "a-set-static-ip": {
                "command": {
                  "Fn::Join": [
                    "",
                    [
                      "powershell.exe -ExecutionPolicy RemoteSigned -Command c:\\cfn\\scripts\\Set-StaticIP.ps1"
                    ]
                  ]
                },
                "waitAfterCompletion": "15"
              },
              "b-run-powershell-RenameComputer-no-reboot": {
                "command": {
                  "Fn::Join": [
                    "",
                    [
                      "powershell.exe Rename-Computer -NewName DC1 -force -restart"
                    ]
                  ]
                },
                "waitAfterCompletion": "forever"
              }
            }
          },
          "installADDS": {
            "commands": {
              "1-install-prereqs": {
                "command": {
                  "Fn::Join": [
                    "",
                    [
                      "powershell.exe -Command \"Install-WindowsFeature AD-Domain-Services, rsat-adds -IncludeAllSubFeature"
                    ]
                  ]
                },
                "waitAfterCompletion": "0"
              },
              "2-install-adds": {
                "command": {
                  "Fn::Join": [
                    "",
                    [
                      "powershell.exe -Command Install-ADDSForest -DomainName ",
                      {
                        "Ref": "DomainDNSName"
                      },
                      " -SafeModeAdministratorPassword (ConvertTo-SecureString '",
                      {
                        "Ref": "AdminPassword"
                      },
                      "' -AsPlainText -Force) -DomainMode Win2012R2 -DomainNetbiosName ",
                      {
                        "Ref": "DomainNetBiosName"
                      },
                      " -ForestMode Win2012R2 -Confirm:$false -Force"
                    ]
                  ]
                },
                "waitAfterCompletion": "forever"
              },
              "3-restart-service": {
                "command": {
                    "Fn::Join": [
                        "",
                        [
                            "powershell.exe -Command Restart-Service NetLogon -EA 0"
                        ]
                    ]
                },
                "waitAfterCompletion": "60"
              },          
              "4-create-adminuser": {
                "command": {
                  "Fn::Join": [
                    "",
                    [
                      "powershell.exe -Command $s = Get-Service -Name ADWS; while ($s.Status -ne 'Running'){ Start-Service ADWS; Start-Sleep 3 }; Start-Sleep 60\n",
                      "powershell.exe -Command $u = New-ADUser -Name ",
                      {
                        "Ref": "DomainAdminUser"
                      },
                      " -UserPrincipalName ",
                      {
                        "Ref": "DomainAdminUser"
                      },
                      "@",
                      {
                        "Ref": "DomainDNSName"
                      },
                      " -AccountPassword (ConvertTo-SecureString '",
                      {
                        "Ref": "AdminPassword"
                      },
                      "' -AsPlainText -Force) -Enabled $true -PasswordNeverExpires $true -PassThru"
                    ]
                  ]
                },
                "waitAfterCompletion": "0"
              },
              "5-update-adminuser": {
                "command": {
                  "Fn::Join": [
                    "",
                    [
                      "powershell.exe -ExecutionPolicy RemoteSigned -Command c:\\cfn\\scripts\\ConvertTo-EnterpriseAdmin.ps1 -Members ",
                      {
                        "Ref": "DomainAdminUser"
                      }
                    ]
                  ]
                },
                "waitAfterCompletion": "0"
              }
            }
          }
        }
      },
      "Properties": {
        "BlockDeviceMappings": [
          {
            "DeviceName": "/dev/sda1",
            "Ebs": {
              "VolumeSize": "40"
            }
          }
        ],
        "ImageId": {
          "Ref": "BaseAmiId"
        },
        "InstanceType": "m4.large",
        "KeyName": {
          "Ref": "KeyName"
        },
        "NetworkInterfaces": [
          {
            "AssociatePublicIpAddress": "false",
            "DeleteOnTermination": "true",
            "DeviceIndex": "0",
            "GroupSet": [
              {
                "Ref": "PrivateSecurityGroup"
              }
            ],
            "PrivateIpAddress": "10.1.3.100",
            "SubnetId": {
              "Ref": "PrivateSubnetAZ1"
            }
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "DC1"
          }
        ],
        "UserData": {
          "Fn::Base64": {
            "Fn::Join": [
              "",
              [
                "<script>\n",
                "cfn-init.exe -v -c config -s ",
                {
                  "Ref": "AWS::StackId"
                },
                " -r DC1",
                " --region ",
                {
                  "Ref": "AWS::Region"
                },
                "\n",
                "</script>\n"
              ]
            ]
          }
        }
      }
    },
    "Server1": {
      "DependsOn" : "DC1",
      "Properties": {
        "ImageId": {
          "Ref": "BaseAmiId"
        },
        "InstanceType": "m4.large",
        "KeyName": {
          "Ref": "KeyName"
        },
        "NetworkInterfaces": [
          {
            "AssociatePublicIpAddress": "true",
            "DeleteOnTermination": "true",
            "DeviceIndex": "0",
            "GroupSet": [
              {
                "Ref": "BastionSecurityGroup"
              }
            ],
            "PrivateIpAddress": "10.1.3.101",
            "SubnetId": {
              "Ref": "PrivateSubnetAZ1"
            }
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "Server1"
          }
        ],
        "UserData": {
          "Fn::Base64": {
            "Fn::Join": [
              "",
              [
                "<powershell>\n",
                "### Change local admin password for consistency...\n",
                "([adsi]\"WinNT://$env:computername/Administrator\").SetPassword('",
                {
                  "Ref": "AdminPassword"
                },
                "') \n",
                "### Join the domain...\n",             
                "$computer = Get-WmiObject -Class Win32_ComputerSystem \n",
                "if ($computer.domain -eq 'WORKGROUP') { \n",
                "  $adapter = Get-NetAdapter -Name 'Ethernet*'\n",
                "  Set-DNSClientServerAddress -InterfaceAlias $adapter.Name -ServerAddresses ('",
                {
                  "Fn::GetAtt": [
                    "DC1",
                    "PrivateIp"
                  ]
                },
                "')\n",
                "  $strNETLOGON='\\\\",
                {
                  "Fn::GetAtt": [
                    "DC1",
                    "PrivateIp"
                  ]
                },
                "\\NETLOGON' \n",
                "  $done=$false \n",
                "  while (!$done) {\n",
                "    Start-Sleep 5 \n",
                "    net use * $strNETLOGON /user:",
                {
                  "Ref": "DomainAdminUser"
                },              
                " '",
                {
                  "Ref": "AdminPassword"
                },
                "'\n",
                "    if (Test-Path $strNETLOGON) {$done=$true} \n",
                "  } \n",
                "  $domain = '",
                {
                  "Ref": "DomainDNSName"
                },
                "'\n",
                "  $password = '",
                {
                  "Ref": "AdminPassword"
                },
                "' | ConvertTo-SecureString -asPlainText -Force \n",
                "  $Administrator = '",
                {
                  "Ref": "DomainNetBiosName"
                },
                "\\",
                {
                  "Ref": "DomainAdminUser"
                },
                "'\n",
                "  $credential = New-Object System.Management.Automation.PSCredential($Administrator,$password) \n",
                "  Add-Computer -DomainName $domain -Credential $credential -restart \n",
                "}\n",
                "</powershell>\n"
              ]
            ]
          }
        }
      },
      "Type": "AWS::EC2::Instance"
    },
    
    "PrivateSecurityGroup": {
      "Properties": {
        "GroupDescription": "This is the security group for Active Directory",
        "SecurityGroupEgress": [
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "tcp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "udp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          },
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "80",
            "IpProtocol": "tcp",
            "ToPort": "80"
          },
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "443",
            "IpProtocol": "tcp",
            "ToPort": "443"
          }
        ],
        "SecurityGroupIngress": [
          {
            "FromPort": "3389",
            "IpProtocol": "tcp",
            "SourceSecurityGroupId": {
              "Ref": "BastionSecurityGroup"
            },
            "ToPort": "3389"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "tcp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "udp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "PrivateSecurityGroup"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::SecurityGroup"
    },
    "BastionSecurityGroup": {
      "Properties": {
        "GroupDescription": "This is the security group for the bastion host",
        "SecurityGroupEgress": [
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "80",
            "IpProtocol": "tcp",
            "ToPort": "80"
          },
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "443",
            "IpProtocol": "tcp",
            "ToPort": "443"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "3389",
            "IpProtocol": "tcp",
            "ToPort": "3389"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "tcp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "udp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          }
        ],
        "SecurityGroupIngress": [
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "3389",
            "IpProtocol": "tcp",
            "ToPort": "3389"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "BastionSecurityGroup"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::SecurityGroup"
    },

    "PublicSubnetAZ1": {
      "Properties": {
        "AvailabilityZone": {
          "Fn::Select": [
            0,
            {
              "Fn::GetAZs": ""
            }
          ]
        },
        "CidrBlock": "10.1.1.0/24",
        "Tags": [
          {
            "Key": "Name",
            "Value": "PublicSubnetAZ1"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::Subnet"
    },
    "PublicSubnetAZ1SubnetAssociation": {
      "Properties": {
        "RouteTableId": {
          "Ref": "DMZRouteTable"
        },
        "SubnetId": {
          "Ref": "PublicSubnetAZ1"
        }
      },
      "Type": "AWS::EC2::SubnetRouteTableAssociation"
    },

    "PrivateSubnetAZ1": {
      "Properties": {
        "AvailabilityZone": {
          "Fn::Select": [
            0,
            {
              "Fn::GetAZs": ""
            }
          ]
        },
        "CidrBlock": "10.1.3.0/24",
        "Tags": [
          {
            "Key": "Name",
            "Value": "PrivateSubnetAZ1"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::Subnet"
    },
    "PrivateSubnetAZ1SubnetAssociation": {
      "Properties": {
        "RouteTableId": {
          "Ref": "InternalRouteTable"
        },
        "SubnetId": {
          "Ref": "PrivateSubnetAZ1"
        }
      },
      "Type": "AWS::EC2::SubnetRouteTableAssociation"
    },
    
    "DMZRouteTable": {
      "Properties": {
        "Tags": [
          {
            "Key": "Application",
            "Value": {
              "Ref": "AWS::StackName"
            }
          },
          {
            "Key": "Name",
            "Value": "DMZRouteTable"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::RouteTable"
    },
    "InternalRouteTable": {
      "Properties": {
        "Tags": [
          {
            "Key": "Name",
            "Value": "InternalRouteTable"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::RouteTable"
    },
    "NATRoute": {
      "Properties": {
        "DestinationCidrBlock": "0.0.0.0/0",
        "NatGatewayId": {
          "Ref": "AWSNat"
        },
        "RouteTableId": {
          "Ref": "InternalRouteTable"
        }
      },
      "Type": "AWS::EC2::Route"
    },
    "PublicRoute": {
      "DependsOn" : "InternetGatewayAttachment",    
      "Properties": {
        "DestinationCidrBlock": "0.0.0.0/0",
        "GatewayId": {
          "Ref": "InternetGateway"
        },
        "RouteTableId": {
          "Ref": "DMZRouteTable"
        }
      },
      "Type": "AWS::EC2::Route"
    },
    "VPC": {
      "Properties": {
        "CidrBlock": "10.1.0.0/16",
        "EnableDnsHostnames": "true",
        "EnableDnsSupport": "true",
        "Tags": [
          {
            "Key": "Name",
            "Value": "VPC"
          }
        ]
      },
      "Type": "AWS::EC2::VPC"
    },
    "InternetGateway": {
      "Properties": {
        "Tags": [
          {
            "Key": "Name",
            "Value": "InternetGateway"
          }
        ]
      },
      "Type": "AWS::EC2::InternetGateway"
    },
    "InternetGatewayAttachment": {
      "Properties": {
        "InternetGatewayId": {
          "Ref": "InternetGateway"
        },
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::VPCGatewayAttachment"
    },
    "AWSNat": {
      "DependsOn": "InternetGatewayAttachment",     
      "Properties": {
        "AllocationId": {
          "Fn::GetAtt": [
            "NatEip",
            "AllocationId"
          ]
        },
        "SubnetId": {
          "Ref": "PublicSubnetAZ1"
        }
      },
      "Type": "AWS::EC2::NatGateway"
    },
    "NatEip": {
      "Properties": {
        "Domain": "vpc"
      },
      "Type": "AWS::EC2::EIP"
    }
  }
}

Notice in Domain-Join1.json that the DC1 resource has the following CreationPolicy attribute:

      "CreationPolicy" : {
        "ResourceSignal" : {
          "Timeout": "PT20M",
          "Count"  : "1"
        }
      },

This CreationPolicy attribute sets a 20-minute timeout. If the instance hasn’t finished launching and running all of its initialization code within 20 minutes, then CloudFormation considers it to have failed. We’ll get to the Metadata section of DC1 in a minute, but the second half of the primary logic is that Server1 has a DependsOn property, like this:

"Server1": {  "DependsOn" : "DC1", ...

This means that CloudFormation will not even launch Server1 until DC1 has the status of CREATE_COMPLETE. Putting this all together, Server1 isn’t ready until all of the following events occur in order:

Now look at the Metadata section of DC1 in Domain-Join1.json. It has code to load a couple of CloudFormation configuration files and a couple of PowerShell scripts on the instance. It also sets a static IP address for the domain controller (recommended practice), and renames the instance to DC1. Then it has commands that install the Active Directory forest and create the domain administrator.

None of the commands in the Metadata section will run unless the UserData section of DC1 includes the command to run cfn-init.exe.

Before you leave this section, here’s a CloudFormation best practice. Before your stack can run cfn-init.exe, the stack must have established outbound Internet access. This means that your NAT gateway, security groups, and routing table resources must all have the status CREATE_COMPLETE. The DC1 resource achieves this with a DependsOn attribute for the NATRoute.

Using cfn-signal and a PowerShell loop

Now consider a better way to do this. The DependsOn attribute on Server1 is the main roadblock in the preceding approach, so let’s remove that. Using Julien’s technique, you can replace it with a PowerShell loop in the UserData section of Server1 that polls a WaitHandle. The WaitHandle is triggered by DC1 when it finishes running all of its initiation logic to install the Active Directory forest. This allows Server1 to launch simultaneously with DC1, but not connect to the domain until DC1 has created the forest.

This diagram shows the compressed sequence of events:

Filename: Domain-Join2.json

{
  "Description": "CloudFormation template for domain join with cfn-signal and PowerShell loop", 
  "Parameters": {
    "KeyName": {
      "MinLength" : 1,
      "Type": "AWS::EC2::KeyPair::KeyName"
    },
    "BaseAmiId": {
      "Default": "ami-bd3ba0aa",
      "Type": "String"
    },
    "DomainAdminUser": {
      "Description": "User name for the account that will be added as Domain Administrator. This is separate from the default \"Administrator\" account",
      "Type": "String",
      "Default": "StackAdmin"
    },
    "AdminPassword": { 
        "NoEcho": "true",
        "Description" : "The Windows administrator account password",
        "Type": "String",
        "MinLength": "8",
        "MaxLength": "41" 
    },
    "DomainDNSName": {
      "Description": "Fully qualified domain name (FQDN) of the forest root domain",
      "Type": "String",
      "Default": "mydomain.local"
    },
    "DomainNetBiosName": {
      "Description": "Netbios name for the domain",
      "Type": "String",
      "Default": "mydomain"
    }
  },
  "Resources": {
    "RootRole": {
        "Type": "AWS::IAM::Role",
        "Properties": {
            "AssumeRolePolicyDocument": {
                "Version" : "2012-10-17",
                "Statement": [ {
                    "Effect": "Allow",
                    "Principal": {
                        "Service": [ "ec2.amazonaws.com" ]
                    },
                "Action": [ "sts:AssumeRole" ]
                } ]
            },
            "Path": "/",
            "Policies": [ {
                "PolicyName": "root",
                "PolicyDocument": {
                    "Version" : "2012-10-17",
                    "Statement": [ {
                        "Effect": "Allow",
                        "Action": "*",
                        "Resource": "*"
                    } ]
                }
            } ]
        }
    },
    "RootInstanceProfile": {
        "Type": "AWS::IAM::InstanceProfile",
        "Properties": {
            "Path": "/",
            "Roles": [ {
                "Ref": "RootRole"
            } ]
        }
    },

    "DomainControllerWaitCondition": {
      "Type": "AWS::CloudFormation::WaitCondition",
      "DependsOn": "DC1",
      "Properties": {
        "Handle": {
          "Ref": "DomainControllerWaitHandle"
        },
        "Timeout": "1100"
      }
    },
    "DomainControllerWaitHandle": {
      "Type": "AWS::CloudFormation::WaitConditionHandle"
    },

    "BastionHost1": {
      "Properties": {
        "ImageId": {
          "Ref": "BaseAmiId"
        },
        "InstanceType": "t2.large",
        "KeyName": {
          "Ref": "KeyName"
        },
        "NetworkInterfaces": [
          {
            "AssociatePublicIpAddress": "true",
            "DeleteOnTermination": "true",
            "DeviceIndex": "0",
            "GroupSet": [
              {
                "Ref": "BastionSecurityGroup"
              }
            ],
            "PrivateIpAddress": "10.1.1.100",
            "SubnetId": {
              "Ref": "PublicSubnetAZ1"
            }
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "BastionHost1"
          }
        ],
        "UserData": {
          "Fn::Base64": {
            "Fn::Join": [
              "",
              [
                "<powershell>\n",
                "### Change local admin password...\n",
                "([adsi]\"WinNT://$env:computername/Administrator\").SetPassword('",
                {
                  "Ref": "AdminPassword"
                },
                "') \n",
                "</powershell>\n"       
              ]
            ]
          }
        }       
      },
      "Type": "AWS::EC2::Instance"
    },  
    "DC1": {    
      "Type": "AWS::EC2::Instance",
      "DependsOn": "NATRoute",    
      "Metadata": {
        "AWS::CloudFormation::Init": {
            "configSets": {
                "config": [
                  "setup",
                  "rename",
                  "installADDS",
                  "finalize"
                ]
            },
            "setup": {
                "files": {
                  "c:\\cfn\\cfn-hup.conf": {
                    "content": {
                      "Fn::Join": [
                        "",
                        [
                          "[main]\n",
                          "stack=",
                          {
                            "Ref": "AWS::StackName"
                          },
                          "\n",
                          "region=",
                          {
                            "Ref": "AWS::Region"
                          },
                          "\n"
                        ]
                      ]
                    }
                  },
                  "c:\\cfn\\hooks.d\\cfn-auto-reloader.conf": {
                    "content": {
                      "Fn::Join": [
                        "",
                        [
                          "[cfn-auto-reloader-hook]\n",
                          "triggers=post.update\n",
                          "path=Resources.DC1.Metadata.AWS::CloudFormation::Init\n",
                          "action=cfn-init.exe -v -c config -s ",
                          {
                            "Ref": "AWS::StackId"
                          },
                          " -r DC1",
                          " --region ",
                          {
                            "Ref": "AWS::Region"
                          },
                          "\n"
                        ]
                      ]
                    }
                  },
                  "c:\\cfn\\scripts\\Set-StaticIP.ps1": {
                    "content": {
                      "Fn::Join": [
                        "",
                        [
                          "$netip = Get-NetIPConfiguration;",
                          "$ipconfig = Get-NetIPAddress | ?{$_.IpAddress -eq $netip.IPv4Address.IpAddress};",
                          "Get-NetAdapter | Set-NetIPInterface -DHCP Disabled;",
                          "Get-NetAdapter | New-NetIPAddress -AddressFamily IPv4 -IPAddress $netip.IPv4Address.IpAddress -PrefixLength $ipconfig.PrefixLength -DefaultGateway $netip.IPv4DefaultGateway.NextHop;",
                          "Get-NetAdapter | Set-DnsClientServerAddress -ServerAddresses $netip.DNSServer.ServerAddresses;",
                          "\n"
                        ]
                      ]
                    }
                  }
                },
                "services": {
                  "windows": {
                    "cfn-hup": {
                      "enabled": "true",
                      "ensureRunning": "true",
                      "files": [
                        "c:\\cfn\\cfn-hup.conf",
                        "c:\\cfn\\hooks.d\\cfn-auto-reloader.conf"
                      ]
                    }
                  }
                },
                "commands": {
                  "a-disable-win-fw": {
                    "command": {
                      "Fn::Join": [
                        "",
                        [
                          "powershell.exe -Command \"Get-NetFirewallProfile | Set-NetFirewallProfile -Enabled False"
                        ]
                      ]
                    },
                    "waitAfterCompletion": "0"
                  }
                }
            },
            "rename": {
                "commands": {
                  "a-set-static-ip": {
                    "command": {
                      "Fn::Join": [
                        "",
                        [
                          "powershell.exe -ExecutionPolicy RemoteSigned -Command c:\\cfn\\scripts\\Set-StaticIP.ps1"
                        ]
                      ]
                    },
                    "waitAfterCompletion": "15"
                  },
                  "b-run-powershell-RenameComputer-no-reboot": {
                    "command": {
                      "Fn::Join": [
                        "",
                        [
                          "powershell.exe Rename-Computer -NewName DC1 -force -restart"
                        ]
                      ]
                    },
                    "waitAfterCompletion": "forever"
                  }
                }
            },
            "installADDS": {
                "commands": {
                    "1-install-prereqs": {
                        "command": {
                          "Fn::Join": [
                            "",
                            [
                              "powershell.exe -Command \"Install-WindowsFeature AD-Domain-Services, rsat-adds -IncludeAllSubFeature"
                            ]
                          ]
                        },
                        "waitAfterCompletion": "0"
                    },
                    "2-install-adds": {
                        "command": {
                          "Fn::Join": [
                            "",
                            [
                              "powershell.exe -Command Install-ADDSForest -DomainName ",
                              {
                                "Ref": "DomainDNSName"
                              },
                              " -SafeModeAdministratorPassword (ConvertTo-SecureString '",
                              {
                                "Ref": "AdminPassword"
                              },
                              "' -AsPlainText -Force) -DomainMode Win2012R2 -DomainNetbiosName ",
                              {
                                "Ref": "DomainNetBiosName"
                              },
                              " -ForestMode Win2012R2 -Confirm:$false -Force"
                            ]
                          ]
                        },
                        "waitAfterCompletion": "forever"
                    },
                    "3-restart-service": {
                        "command": {
                        "Fn::Join": [
                            "",
                            [
                                "powershell.exe -Command Restart-Service NetLogon -EA 0"
                            ]
                        ]
                        },
                        "waitAfterCompletion": "20"
                    },    
                    "4-start-ADWS": {
                        "command": {
                          "Fn::Join": [
                            "",
                            [
                              "powershell.exe -Command $s = Get-Service -Name ADWS; while ($s.Status -ne 'Running'){ Start-Service ADWS; Start-Sleep 3 }"
                            ]
                          ]
                        },
                        "waitAfterCompletion": "30"
                    },                    
                    "5-create-adminuser": {
                        "command": {
                        "Fn::Join": [
                        "",
                        [
                          "powershell.exe -Command $u = New-ADUser ",
                          {
                            "Ref": "DomainAdminUser"
                          },
                          " -SamAccountName ",
                          {
                            "Ref": "DomainAdminUser"
                          },                  
                          " -UserPrincipalName ",
                          {
                            "Ref": "DomainAdminUser"
                          },
                          "@",
                          {
                            "Ref": "DomainDNSName"
                          },
                          " -AccountPassword (ConvertTo-SecureString '",
                          {
                            "Ref": "AdminPassword"
                          },
                          "' -AsPlainText -Force) -Enabled $true -PasswordNeverExpires $true -PassThru; Add-ADGroupMember -Identity 'domain admins' -Members $u"
                        ]
                      ]
                    },
                    "waitAfterCompletion": "0"
                }
              }
            },
            "finalize": {
                "commands": {
                    "a-signal-success": {
                        "command": {
                        "Fn::Join": [
                        "",
                        [
                          "cfn-signal.exe -e 0 ",
                          {
                            "Fn::Base64": {
                              "Ref": "DomainControllerWaitHandle"
                            }
                          },
                          ""
                        ]
                        ]
                        }
                    }
                }
            }
        }
      },
      "Properties": {
            "BlockDeviceMappings": [
              {
                "DeviceName": "/dev/sda1",
                "Ebs": {
                  "VolumeSize": "40"
                }
              }
            ],
            "ImageId": {
              "Ref": "BaseAmiId"
            },
            "InstanceType": "m4.large",
            "KeyName": {
              "Ref": "KeyName"
            },
            "NetworkInterfaces": [
              {
                "AssociatePublicIpAddress": "false",
                "DeleteOnTermination": "true",
                "DeviceIndex": "0",
                "GroupSet": [
                  {
                    "Ref": "PrivateSecurityGroup"
                  }
                ],
                "PrivateIpAddress": "10.1.3.100",
                "SubnetId": {
                  "Ref": "PrivateSubnetAZ1"
                }
              }
            ],
            "Tags": [
              {
                "Key": "Name",
                "Value": "DC1"
              }
            ],
            "UserData": {
              "Fn::Base64": {
                "Fn::Join": [
                  "",
                  [
                    "<script>\n",
                    "cfn-init.exe -v -c config -s ",
                    {
                      "Ref": "AWS::StackId"
                    },
                    " -r DC1",
                    " --region ",
                    {
                      "Ref": "AWS::Region"
                    },
                    "\n",
                    "</script>\n"
                  ]
                ]
              }
            }
        }
    },
    "Server1": {
      "Properties": {
        "ImageId": {
          "Ref": "BaseAmiId"
        },
        "InstanceType": "m4.large",
        "IamInstanceProfile" : {"Ref" : "RootInstanceProfile"},
        "KeyName": {
          "Ref": "KeyName"
        },
        "NetworkInterfaces": [
          {
            "AssociatePublicIpAddress": "true",
            "DeleteOnTermination": "true",
            "DeviceIndex": "0",
            "GroupSet": [
              {
                "Ref": "BastionSecurityGroup"
              }
            ],
            "PrivateIpAddress": "10.1.3.101",
            "SubnetId": {
              "Ref": "PrivateSubnetAZ1"
            }
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "Server1"
          }
        ],
        "UserData": {
          "Fn::Base64": {
            "Fn::Join": [
              "",
              [
                "<powershell>\n",
                "### Change local admin password...\n",
                "([adsi]\"WinNT://$env:computername/Administrator\").SetPassword('",
                {
                  "Ref": "AdminPassword"
                },
                "') \n",
                "### Wait for DC to be fully available...\n",               
                "$resource = 'DomainControllerWaitCondition'\n",
                "$region = '",
                { "Ref": "AWS::Region" },
                "'\n$stack = '",
                { "Ref": "AWS::StackId" },
                "'\n$output = (Get-CFNStackResources -StackName $stack -LogicalResourceId $resource -Region $region)\n",
                "while (($output -eq $null) -or ($output.ResourceStatus -ne 'CREATE_COMPLETE') -and ($output.ResourceStatus -ne 'UPDATE_COMPLETE')) {\n",
                "    Start-Sleep 5\n",
                "    $output = (Get-CFNStackResources -StackName $stack -LogicalResourceId $resource -Region $region)\n",
                "}\n",
                "### Join the domain...\n",             
                "$computer = Get-WmiObject -Class Win32_ComputerSystem \n",
                "if ($computer.domain -eq 'WORKGROUP') { \n",
                "  $adapter = Get-NetAdapter -Name 'Ethernet*'\n",
                "  Set-DNSClientServerAddress -InterfaceAlias $adapter.Name -ServerAddresses ('",
                {
                  "Fn::GetAtt": [
                    "DC1",
                    "PrivateIp"
                  ]
                },
                "')\n",
                "  $domain = '",
                {
                  "Ref": "DomainDNSName"
                },
                "'\n",
                "  $password = '",
                {
                  "Ref": "AdminPassword"
                },
                "' | ConvertTo-SecureString -asPlainText -Force \n",
                "  $Administrator = '",
                {
                  "Ref": "DomainNetBiosName"
                },
                "\\",
                {
                  "Ref": "DomainAdminUser"
                },
                "'\n",
                "  $credential = New-Object System.Management.Automation.PSCredential($Administrator,$password) \n",
                "  Add-Computer -DomainName $domain -Credential $credential -restart \n",
                "}\n",
                "</powershell>\n"
              ]
            ]
          }
        }
      },
      "Type": "AWS::EC2::Instance"
    },
    
    "PrivateSecurityGroup": {
      "Properties": {
        "GroupDescription": "This is the security group for Active Directory",
        "SecurityGroupEgress": [
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "tcp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "udp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          },
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "80",
            "IpProtocol": "tcp",
            "ToPort": "80"
          },
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "443",
            "IpProtocol": "tcp",
            "ToPort": "443"
          }
        ],
        "SecurityGroupIngress": [
          {
            "FromPort": "3389",
            "IpProtocol": "tcp",
            "SourceSecurityGroupId": {
              "Ref": "BastionSecurityGroup"
            },
            "ToPort": "3389"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "tcp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "udp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "PrivateSecurityGroup"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::SecurityGroup"
    },
    "BastionSecurityGroup": {
      "Properties": {
        "GroupDescription": "This is the security group for the bastion host",
        "SecurityGroupEgress": [
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "80",
            "IpProtocol": "tcp",
            "ToPort": "80"
          },
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "443",
            "IpProtocol": "tcp",
            "ToPort": "443"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "3389",
            "IpProtocol": "tcp",
            "ToPort": "3389"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "tcp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "udp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          }
        ],
        "SecurityGroupIngress": [
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "3389",
            "IpProtocol": "tcp",
            "ToPort": "3389"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "BastionSecurityGroup"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::SecurityGroup"
    },

    "PublicSubnetAZ1": {
      "Properties": {
        "AvailabilityZone": {
          "Fn::Select": [
            0,
            {
              "Fn::GetAZs": ""
            }
          ]
        },
        "CidrBlock": "10.1.1.0/24",
        "Tags": [
          {
            "Key": "Name",
            "Value": "PublicSubnetAZ1"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::Subnet"
    },
    "PublicSubnetAZ1SubnetAssociation": {
      "Properties": {
        "RouteTableId": {
          "Ref": "DMZRouteTable"
        },
        "SubnetId": {
          "Ref": "PublicSubnetAZ1"
        }
      },
      "Type": "AWS::EC2::SubnetRouteTableAssociation"
    },

    "PrivateSubnetAZ1": {
      "Properties": {
        "AvailabilityZone": {
          "Fn::Select": [
            0,
            {
              "Fn::GetAZs": ""
            }
          ]
        },
        "CidrBlock": "10.1.3.0/24",
        "Tags": [
          {
            "Key": "Name",
            "Value": "PrivateSubnetAZ1"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::Subnet"
    },
    "PrivateSubnetAZ1SubnetAssociation": {
      "Properties": {
        "RouteTableId": {
          "Ref": "InternalRouteTable"
        },
        "SubnetId": {
          "Ref": "PrivateSubnetAZ1"
        }
      },
      "Type": "AWS::EC2::SubnetRouteTableAssociation"
    },
    
    "DMZRouteTable": {
      "Properties": {
        "Tags": [
          {
            "Key": "Application",
            "Value": {
              "Ref": "AWS::StackName"
            }
          },
          {
            "Key": "Name",
            "Value": "DMZRouteTable"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::RouteTable"
    },
    "InternalRouteTable": {
      "Properties": {
        "Tags": [
          {
            "Key": "Name",
            "Value": "InternalRouteTable"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::RouteTable"
    },
    "NATRoute": {
      "Properties": {
        "DestinationCidrBlock": "0.0.0.0/0",
        "NatGatewayId": {
          "Ref": "AWSNat"
        },
        "RouteTableId": {
          "Ref": "InternalRouteTable"
        }
      },
      "Type": "AWS::EC2::Route"
    },
    "PublicRoute": {
      "DependsOn" : "InternetGatewayAttachment",        
      "Properties": {
        "DestinationCidrBlock": "0.0.0.0/0",
        "GatewayId": {
          "Ref": "InternetGateway"
        },
        "RouteTableId": {
          "Ref": "DMZRouteTable"
        }
      },
      "Type": "AWS::EC2::Route"
    },
    "VPC": {
      "Properties": {
        "CidrBlock": "10.1.0.0/16",
        "EnableDnsHostnames": "true",
        "EnableDnsSupport": "true",
        "Tags": [
          {
            "Key": "Name",
            "Value": "VPC"
          }
        ]
      },
      "Type": "AWS::EC2::VPC"
    },
    "InternetGateway": {
      "Properties": {
        "Tags": [
          {
            "Key": "Name",
            "Value": "InternetGateway"
          }
        ]
      },
      "Type": "AWS::EC2::InternetGateway"
    },
    "InternetGatewayAttachment": {
      "Properties": {
        "InternetGatewayId": {
          "Ref": "InternetGateway"
        },
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::VPCGatewayAttachment"
    },
    "AWSNat": {
      "DependsOn": "InternetGatewayAttachment",     
      "Properties": {
        "AllocationId": {
          "Fn::GetAtt": [
            "NatEip",
            "AllocationId"
          ]
        },
        "SubnetId": {
          "Ref": "PublicSubnetAZ1"
        }
      },
      "Type": "AWS::EC2::NatGateway"
    },
    "NatEip": {
      "Properties": {
        "Domain": "vpc"
      },
      "Type": "AWS::EC2::EIP"
    }
  }
}

Here’s the WaitCondition and the WaitHandle code in Domain-Join2.json (they go together in CloudFormation):

    "DomainControllerWaitCondition": {
      "Type": "AWS::CloudFormation::WaitCondition",
      "DependsOn": "DC1",
      "Properties": {
        "Handle": {
          "Ref": "DomainControllerWaitHandle"
        },
        "Timeout": "1100"
      }
    },
    "DomainControllerWaitHandle": {
      "Type": "AWS::CloudFormation::WaitConditionHandle"
    },

Now look at the finalize section in the DC1 metadata. It runs cfn-signal.exe to trigger the WaitHandle:

        "finalize": {
                "commands": {
                        "a-signal-success": {
                                "command": {
                                "Fn::Join": [
                                "",
                                [
                                 "cfn-signal.exe -e 0 ",
                                 {
                                        "Fn::Base64": {
                                         "Ref": "DomainControllerWaitHandle"
                                        }
                                 },
                                 ""
                                ]
                                ]
                                }
                        }
                }
        }

The final piece is the PowerShell loop in the Server1 UserData section. It polls the WaitHandle until it is marked as CREATE_COMPLETE by cfn-signal:

"'\n$output = (Get-CFNStackResources -StackName $stack -LogicalResourceId $resource -Region $region)\n",
"while (($output -eq $null) -or ($output.ResourceStatus -ne 'CREATE_COMPLETE') -and ($output.ResourceStatus -ne 'UPDATE_COMPLETE')) {\n",
"    Start-Sleep 5\n",
"    $output = (Get-CFNStackResources -StackName $stack -LogicalResourceId $resource -Region $region)\n",
"}\n",

Unfortunately, the syntax of the PowerShell code inside JSON gets a bit tricky with respect to double-quoted strings and newlines. When you develop PowerShell inside CloudFormation, you need to login to the instance and examine the PowerShell results in cfn-init.log. (For more details, see the tip at the end of this section.)

The while loop uses the AWS Tools for PowerShell cmdlet Get-CFNStackResources. Because this is an AWS operation, not a Windows cmdlet, you must provide AWS credentials to PowerShell. The best way to do this is to launch the Server1 instance with an IamInstanceProfile property. You’ll notice that Domain-Join2.json and Domain-Join3.json create an IAM::Role resource, then inject that role into an IAM::InstanceProfile resource, then specify that InstanceProfile in Server1’s IamInstanceProfile property.

One more tip: Remember when I said that your stack needs outbound Internet access before it can run cfn-init.exe? The same is true if your stack uses cfn-signal.exe, or Get-CFNStackResources, or many other AWS cmdlets.

Using a custom AMI that has the Active Directory role preinstalled

Using cfn-signal definitely speeds things up because it enables Server1 to launch simultaneously with DC1. But the whole process is still delayed waiting for the Active Directory role to be installed, the forest to be created, and DC1 to be rebooted. If you are running this repeatedly in a dev/test environment, you want it to be as fast as possible. To do that, you can create the domain controller and make a custom AMI of it. Then specify that AMI ID when launching DC1.

Here’s the optimized sequence of events:

To use this technique, you need to make the following changes to the previous approach:

  • Run Domain-Join3-Create.json (see below) in CloudFormation. It builds only a domain controller called DC1.
  • Connect to DC1 using RDP. (As described in the preceding section, first remotely connect to BastionHost1 as an administrator, then remotely connect to DC1 as mydomain\StackAdmin.)
  • On DC1, run Ec2configSettings.exe to enable execution of the UserData section. See screenshot below. (This updates C:\Program Files\Amazon\Ec2ConfigService\Settings\config.xml.)

Enable UserData execution in Ec2ConfigSettings.exe

  • Stop the DC1 instance in the EC2 Dashboard (a best practice before making an image). Wait for the instance state to change to stopped , and then create an image of it.
  • In CloudFormation, delete the stack.
  • Open Domain-Join3.json and edit the DCAmiId parameter so that it matches the custom AMI that you just created. You’ll notice that I removed a large chunk of code that installs the Windows feature and creates the forest from this template because that logic already ran in Domain-Join3-Create.json.
  • Finally, run Domain-Join3.json in CloudFormation whenever you want to deploy the complete sample infrastructure. This is the fastest way I’ve found to do this, but I’d love to hear your ideas if you have improvements.

Filename: Domain-Join3-Create.json

{
  "Description": "CloudFormation template to create domain controller for AMI", 
  "Parameters": {
    "KeyName": {
      "MinLength" : 1,  
      "Type": "AWS::EC2::KeyPair::KeyName"
    },
    "BaseAmiId": {
      "Default": "ami-bd3ba0aa",
      "Type": "String"
    },
    "DomainAdminUser": {
      "Description": "User name for the account that will be added as Domain Administrator. This is separate from the default \"Administrator\" account",
      "Type": "String",
      "Default": "StackAdmin"
    },
    "AdminPassword": {
        "NoEcho": "true",
        "Description" : "The Windows administrator account password",
        "Type": "String",
        "MinLength": "8",
        "MaxLength": "41" 
    },
    "DomainDNSName": {
      "Description": "Fully qualified domain name (FQDN) of the forest root domain",
      "Type": "String",
      "Default": "mydomain.local"
    },
    "DomainNetBiosName": {
      "Description": "Netbios name for the domain",
      "Type": "String",
      "Default": "mydomain"
    }
  },
  "Resources": {

    "BastionHost1": {
      "Properties": {
        "ImageId": {
          "Ref": "BaseAmiId"
        },
        "InstanceType": "t2.large",
        "KeyName": {
          "Ref": "KeyName"
        },
        "NetworkInterfaces": [
          {
            "AssociatePublicIpAddress": "true",
            "DeleteOnTermination": "true",
            "DeviceIndex": "0",
            "GroupSet": [
              {
                "Ref": "BastionSecurityGroup"
              }
            ],
            "PrivateIpAddress": "10.1.1.100",
            "SubnetId": {
              "Ref": "PublicSubnetAZ1"
            }
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "BastionHost1"
          }
        ],
        "UserData": {
          "Fn::Base64": {
            "Fn::Join": [
              "",
              [
                "<powershell>\n",
                "### Change local admin password...\n",
                "([adsi]\"WinNT://$env:computername/Administrator\").SetPassword('",
                {
                  "Ref": "AdminPassword"
                },
                "') \n",
                "</powershell>\n"               
              ]
            ]
          }
        }       
      },
      "Type": "AWS::EC2::Instance"
    },  
    "DC1": {    
      "Type": "AWS::EC2::Instance",
      "DependsOn": "NATRoute",    
      "Metadata": {
        "AWS::CloudFormation::Init": {
            "configSets": {
                "config": [
                  "setup",
                  "rename",
                  "installADDS"
                ]
            },
            "setup": {
                "files": {
                  "c:\\cfn\\scripts\\Set-StaticIP.ps1": {
                    "content": {
                      "Fn::Join": [
                        "",
                        [
                          "$netip = Get-NetIPConfiguration;",
                          "$ipconfig = Get-NetIPAddress | ?{$_.IpAddress -eq $netip.IPv4Address.IpAddress};",
                          "Get-NetAdapter | Set-NetIPInterface -DHCP Disabled;",
                          "Get-NetAdapter | New-NetIPAddress -AddressFamily IPv4 -IPAddress $netip.IPv4Address.IpAddress -PrefixLength $ipconfig.PrefixLength -DefaultGateway $netip.IPv4DefaultGateway.NextHop;",
                          "Get-NetAdapter | Set-DnsClientServerAddress -ServerAddresses $netip.DNSServer.ServerAddresses;",
                          "\n"
                        ]
                      ]
                    }
                  }
                },
                "commands": {
                  "a-disable-win-fw": {
                    "command": {
                      "Fn::Join": [
                        "",
                        [
                          "powershell.exe -Command \"Get-NetFirewallProfile | Set-NetFirewallProfile -Enabled False"
                        ]
                      ]
                    },
                    "waitAfterCompletion": "0"
                  }
                }
            },
            "rename": {
                "commands": {
                  "a-set-static-ip": {
                    "command": {
                      "Fn::Join": [
                        "",
                        [
                          "powershell.exe -ExecutionPolicy RemoteSigned -Command c:\\cfn\\scripts\\Set-StaticIP.ps1"
                        ]
                      ]
                    },
                    "waitAfterCompletion": "15"
                  },
                  "b-run-powershell-RenameComputer-no-reboot": {
                    "command": {
                      "Fn::Join": [
                        "",
                        [
                          "powershell.exe Rename-Computer -NewName DC1 -force -restart"
                        ]
                      ]
                    },
                    "waitAfterCompletion": "forever"
                  }
                }
            },
            "installADDS": {
              "commands": {
                "1-install-prereqs": {
                    "command": {
                      "Fn::Join": [
                        "",
                        [
                          "powershell.exe -Command \"Install-WindowsFeature AD-Domain-Services, rsat-adds -IncludeAllSubFeature"
                        ]
                      ]
                    },
                    "waitAfterCompletion": "0"
                },          
                "2-install-adds": {
                    "command": {
                      "Fn::Join": [
                        "",
                        [
                          "powershell.exe -Command Install-ADDSForest -DomainName ",
                          {
                            "Ref": "DomainDNSName"
                          },
                          " -SafeModeAdministratorPassword (ConvertTo-SecureString '",
                          {
                            "Ref": "AdminPassword"
                          },
                          "' -AsPlainText -Force) -DomainMode Win2012R2 -DomainNetbiosName ",
                          {
                            "Ref": "DomainNetBiosName"
                          },
                          " -ForestMode Win2012R2 -Confirm:$false -Force"
                        ]
                      ]
                    },
                    "waitAfterCompletion": "forever"
                },              
                "3-start-ADWS": {
                    "command": {
                      "Fn::Join": [
                        "",
                        [
                          "powershell.exe -Command $s = Get-Service -Name ADWS; while ($s.Status -ne 'Running'){ Start-Service ADWS; Start-Sleep 3 }"
                        ]
                      ]
                    },
                    "waitAfterCompletion": "30"
                },                  
                "4-create-adminuser": {
                    "command": {
                        "Fn::Join": [
                        "",
                        [
                          "powershell.exe -Command $u = New-ADUser ",
                          {
                            "Ref": "DomainAdminUser"
                          },
                          " -SamAccountName ",
                          {
                            "Ref": "DomainAdminUser"
                          },                  
                          " -UserPrincipalName ",
                          {
                            "Ref": "DomainAdminUser"
                          },
                          "@",
                          {
                            "Ref": "DomainDNSName"
                          },
                          " -AccountPassword (ConvertTo-SecureString '",
                          {
                            "Ref": "AdminPassword"
                          },
                          "' -AsPlainText -Force) -Enabled $true -PasswordNeverExpires $true -PassThru; Add-ADGroupMember -Identity 'domain admins' -Members $u"
                        ]
                      ]
                    },
                    "waitAfterCompletion": "0"
                }               
              }
            }
        }
      },
      "Properties": {
            "BlockDeviceMappings": [
              {
                "DeviceName": "/dev/sda1",
                "Ebs": {
                  "VolumeSize": "40"
                }
              }
            ],
            "ImageId": {
              "Ref": "BaseAmiId"
            },
            "InstanceType": "m4.large",
            "KeyName": {
              "Ref": "KeyName"
            },
            "NetworkInterfaces": [
              {
                "AssociatePublicIpAddress": "false",
                "DeleteOnTermination": "true",
                "DeviceIndex": "0",
                "GroupSet": [
                  {
                    "Ref": "PrivateSecurityGroup"
                  }
                ],
                "PrivateIpAddress": "10.1.3.100",
                "SubnetId": {
                  "Ref": "PrivateSubnetAZ1"
                }
              }
            ],
            "Tags": [
              {
                "Key": "Name",
                "Value": "DC1"
              }
            ],
            "UserData": {
              "Fn::Base64": {
                "Fn::Join": [
                  "",
                  [
                    "<script>\n",
                    "cfn-init.exe -v -c config -s ",
                    {
                      "Ref": "AWS::StackId"
                    },
                    " -r DC1",
                    " --region ",
                    {
                      "Ref": "AWS::Region"
                    },
                    "\n",
                    "</script>\n"
                  ]
                ]
              }
            }
        }
    },
    
    "PrivateSecurityGroup": {
      "Properties": {
        "GroupDescription": "This is the security group for Active Directory",
        "SecurityGroupEgress": [
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "tcp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "udp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          },
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "80",
            "IpProtocol": "tcp",
            "ToPort": "80"
          },
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "443",
            "IpProtocol": "tcp",
            "ToPort": "443"
          }
        ],
        "SecurityGroupIngress": [
          {
            "FromPort": "3389",
            "IpProtocol": "tcp",
            "SourceSecurityGroupId": {
              "Ref": "BastionSecurityGroup"
            },
            "ToPort": "3389"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "tcp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "udp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "PrivateSecurityGroup"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::SecurityGroup"
    },
    "BastionSecurityGroup": {
      "Properties": {
        "GroupDescription": "This is the security group for the bastion host",
        "SecurityGroupEgress": [
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "80",
            "IpProtocol": "tcp",
            "ToPort": "80"
          },
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "443",
            "IpProtocol": "tcp",
            "ToPort": "443"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "3389",
            "IpProtocol": "tcp",
            "ToPort": "3389"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "tcp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "udp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          }
        ],
        "SecurityGroupIngress": [
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "3389",
            "IpProtocol": "tcp",
            "ToPort": "3389"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "BastionSecurityGroup"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::SecurityGroup"
    },

    "PublicSubnetAZ1": {
      "Properties": {
        "AvailabilityZone": {
          "Fn::Select": [
            0,
            {
              "Fn::GetAZs": ""
            }
          ]
        },
        "CidrBlock": "10.1.1.0/24",
        "Tags": [
          {
            "Key": "Name",
            "Value": "PublicSubnetAZ1"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::Subnet"
    },
    "PublicSubnetAZ1SubnetAssociation": {
      "Properties": {
        "RouteTableId": {
          "Ref": "DMZRouteTable"
        },
        "SubnetId": {
          "Ref": "PublicSubnetAZ1"
        }
      },
      "Type": "AWS::EC2::SubnetRouteTableAssociation"
    },

    "PrivateSubnetAZ1": {
      "Properties": {
        "AvailabilityZone": {
          "Fn::Select": [
            0,
            {
              "Fn::GetAZs": ""
            }
          ]
        },
        "CidrBlock": "10.1.3.0/24",
        "Tags": [
          {
            "Key": "Name",
            "Value": "PrivateSubnetAZ1"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::Subnet"
    },
    "PrivateSubnetAZ1SubnetAssociation": {
      "Properties": {
        "RouteTableId": {
          "Ref": "InternalRouteTable"
        },
        "SubnetId": {
          "Ref": "PrivateSubnetAZ1"
        }
      },
      "Type": "AWS::EC2::SubnetRouteTableAssociation"
    },
    
    "DMZRouteTable": {
      "Properties": {
        "Tags": [
          {
            "Key": "Application",
            "Value": {
              "Ref": "AWS::StackName"
            }
          },
          {
            "Key": "Name",
            "Value": "DMZRouteTable"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::RouteTable"
    },
    "InternalRouteTable": {
      "Properties": { 
        "Tags": [
          {
            "Key": "Name",
            "Value": "InternalRouteTable"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::RouteTable"
    },
    "NATRoute": {
      "DependsOn" : "InternetGatewayAttachment",        
      "Properties": {
        "DestinationCidrBlock": "0.0.0.0/0",
        "NatGatewayId": {
          "Ref": "AWSNat"
        },
        "RouteTableId": {
          "Ref": "InternalRouteTable"
        }
      },
      "Type": "AWS::EC2::Route"
    },
    "PublicRoute": {
      "Properties": {
        "DestinationCidrBlock": "0.0.0.0/0",
        "GatewayId": {
          "Ref": "InternetGateway"
        },
        "RouteTableId": {
          "Ref": "DMZRouteTable"
        }
      },
      "Type": "AWS::EC2::Route"
    },
    "VPC": {
      "Properties": {
        "CidrBlock": "10.1.0.0/16",
        "EnableDnsHostnames": "true",
        "EnableDnsSupport": "true",
        "Tags": [
          {
            "Key": "Name",
            "Value": "VPC"
          }
        ]
      },
      "Type": "AWS::EC2::VPC"
    },
    "InternetGateway": {
      "Properties": {
        "Tags": [
          {
            "Key": "Name",
            "Value": "InternetGateway"
          }
        ]
      },
      "Type": "AWS::EC2::InternetGateway"
    },
    "InternetGatewayAttachment": {
      "Properties": {
        "InternetGatewayId": {
          "Ref": "InternetGateway"
        },
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::VPCGatewayAttachment"
    },
    "AWSNat": {
      "DependsOn": "InternetGatewayAttachment",     
      "Properties": {
        "AllocationId": {
          "Fn::GetAtt": [
            "NatEip",
            "AllocationId"
          ]
        },
        "SubnetId": {
          "Ref": "PublicSubnetAZ1"
        }
      },
      "Type": "AWS::EC2::NatGateway"
    },
    "NatEip": {
      "Properties": {
        "Domain": "vpc"
      },
      "Type": "AWS::EC2::EIP"
    }
  }
}

Filename: Domain-Join3.json

{
  "Description": "CloudFormation template for domain join with pre-created AMI", 
  "Parameters": {
    "KeyName": {
      "MinLength" : 1,
      "Type": "AWS::EC2::KeyPair::KeyName"
    },
    "BaseAmiId": {
      "Default": "ami-79dc1b14",
      "Type": "String"
    },
    "DCAmiId": {
      "Default": "ami-67284570",
      "Type": "String"
    },  
    "DomainAdminUser": {
      "Description": "User name for the account that will be added as Domain Administrator. This is separate from the default \"Administrator\" account",
      "Type": "String",
      "Default": "StackAdmin"
    },
    "AdminPassword": {
        "NoEcho": "true",
        "Description" : "The Windows administrator account password",
        "Type": "String",
        "MinLength": "8",
        "MaxLength": "41" 
    },
    "DomainDNSName": {
      "Description": "Fully qualified domain name (FQDN) of the forest root domain",
      "Type": "String",
      "Default": "mydomain.local"
    },
    "DomainNetBiosName": {
      "Description": "Netbios name for the domain",
      "Type": "String",
      "Default": "mydomain"
    }
  },
  "Resources": {
    "RootRole": {
        "Type": "AWS::IAM::Role",
        "Properties": {
            "AssumeRolePolicyDocument": {
                "Version" : "2012-10-17",
                "Statement": [ {
                    "Effect": "Allow",
                    "Principal": {
                        "Service": [ "ec2.amazonaws.com" ]
                    },
                "Action": [ "sts:AssumeRole" ]
                } ]
            },
            "Path": "/",
            "Policies": [ {
                "PolicyName": "root",
                "PolicyDocument": {
                    "Version" : "2012-10-17",
                    "Statement": [ {
                        "Effect": "Allow",
                        "Action": "*",
                        "Resource": "*"
                    } ]
                }
            } ]     
        }
    },
    "RootInstanceProfile": {
        "Type": "AWS::IAM::InstanceProfile",
        "Properties": {
            "Path": "/",
            "Roles": [{
                "Ref": "RootRole"
            } ]
        }
    },  
    "DomainControllerWaitCondition": {
      "Type": "AWS::CloudFormation::WaitCondition",
      "DependsOn": "DC1",     
      "Properties": {
        "Handle": {
          "Ref": "DomainControllerWaitHandle"
        },
        "Timeout": "1100"
      }
    },
    "DomainControllerWaitHandle": {
      "Type": "AWS::CloudFormation::WaitConditionHandle"
    },

    "BastionHost1": {
      "Properties": {
        "ImageId": {
          "Ref": "BaseAmiId"
        },
        "InstanceType": "t2.large",
        "KeyName": {
          "Ref": "KeyName"
        },
        "NetworkInterfaces": [
          {
            "AssociatePublicIpAddress": "true",
            "DeleteOnTermination": "true",
            "DeviceIndex": "0",
            "GroupSet": [
              {
                "Ref": "BastionSecurityGroup"
              }
            ],
            "PrivateIpAddress": "10.1.1.100",
            "SubnetId": {
              "Ref": "PublicSubnetAZ1"
            }
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "BastionHost1"
          }
        ],
        "UserData": {
          "Fn::Base64": {
            "Fn::Join": [
              "",
              [
                "<powershell>\n",
                "### Change local admin password...\n",
                "([adsi]\"WinNT://$env:computername/Administrator\").SetPassword('",
                {
                  "Ref": "AdminPassword"
                },
                "') \n",
                "</powershell>\n"       
              ]
            ]
          }
        }       
      },
      "Type": "AWS::EC2::Instance"
    },  
    "DC1": {    
        "Type": "AWS::EC2::Instance",
        "DependsOn": "NATRoute",
        "Metadata": {
            "AWS::CloudFormation::Init": {
                "configSets": {
                    "config": [
                      "setup",
                      "rename",
                      "finalize"
                    ]
                },
                "setup": {
                    "files": {
                      "c:\\cfn\\scripts\\Set-StaticIP.ps1": {
                        "content": {
                          "Fn::Join": [
                            "",
                            [
                              "$netip = Get-NetIPConfiguration;",
                              "$ipconfig = Get-NetIPAddress | ?{$_.IpAddress -eq $netip.IPv4Address.IpAddress};",
                              "Get-NetAdapter | Set-NetIPInterface -DHCP Disabled;",
                              "Get-NetAdapter | New-NetIPAddress -AddressFamily IPv4 -IPAddress $netip.IPv4Address.IpAddress -PrefixLength $ipconfig.PrefixLength -DefaultGateway $netip.IPv4DefaultGateway.NextHop;",
                              "Get-NetAdapter | Set-DnsClientServerAddress -ServerAddresses $netip.DNSServer.ServerAddresses;",
                              "\n"
                            ]
                          ]
                        }
                      }
                    }
                },
                "rename": {
                    "commands": {
                      "a-set-static-ip": {
                        "command": {
                          "Fn::Join": [
                            "",
                            [
                              "powershell.exe -ExecutionPolicy RemoteSigned -Command c:\\cfn\\scripts\\Set-StaticIP.ps1"
                            ]
                          ]
                        },
                        "waitAfterCompletion": "15"
                      }                               
                    }
                },
                "finalize": {
                    "commands": {
                        "a-signal-success": {
                            "command": {
                                "Fn::Join": [
                                "",
                                [
                                  "cfn-signal.exe -e 0 ",
                                  {
                                    "Fn::Base64": {
                                      "Ref": "DomainControllerWaitHandle"
                                    }
                                  },
                                  ""
                                ]
                                ]
                            }
                        }
                    }
                }
            }
        },
        "Properties": {
            "BlockDeviceMappings": [
              {
                "DeviceName": "/dev/sda1",
                "Ebs": {
                  "VolumeSize": "40"
                }
              }
            ],
            "ImageId": {
              "Ref": "DCAmiId"
            },
            "InstanceType": "m4.large",
            "KeyName": {
              "Ref": "KeyName"
            },
            "NetworkInterfaces": [
              {
                "AssociatePublicIpAddress": "false",
                "DeleteOnTermination": "true",
                "DeviceIndex": "0",
                "GroupSet": [
                  {
                    "Ref": "PrivateSecurityGroup"
                  }
                ],
                "PrivateIpAddress": "10.1.3.100",
                "SubnetId": {
                  "Ref": "PrivateSubnetAZ1"
                }
              }
            ],
            "Tags": [
              {
                "Key": "Name",
                "Value": "DC1"
              }
            ],
            "UserData": {
              "Fn::Base64": {
                "Fn::Join": [
                  "",
                  [
                    "<script>\n",
                    "cfn-init.exe -v -c config -s ",
                    {
                      "Ref": "AWS::StackId"
                    },
                    " -r DC1",
                    " --region ",
                    {
                      "Ref": "AWS::Region"
                    },
                    "\n",
                    "</script>\n"
                  ]
                ]
              }
            }
        }
    },
    "Server1": {
      "Properties": {
        "IamInstanceProfile" : {"Ref" : "RootInstanceProfile"},
        "ImageId": {
          "Ref": "BaseAmiId"
        },
        "InstanceType": "m4.large",
    
        "KeyName": {
          "Ref": "KeyName"
        },
        "NetworkInterfaces": [
          {
            "AssociatePublicIpAddress": "true",
            "DeleteOnTermination": "true",
            "DeviceIndex": "0",
            "GroupSet": [
              {
                "Ref": "BastionSecurityGroup"
              }
            ],
            "PrivateIpAddress": "10.1.3.101",
            "SubnetId": {
              "Ref": "PrivateSubnetAZ1"
            }
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "Server1"
          }
        ],
        "UserData": {
          "Fn::Base64": {
            "Fn::Join": [
              "",
              [
                "<powershell>\n",
                "### Change local admin password...\n",
                "([adsi]\"WinNT://$env:computername/Administrator\").SetPassword('",
                {
                  "Ref": "AdminPassword"
                },
                "') \n",
                "### Wait for DC to be fully available...\n",               
                "$resource = 'DomainControllerWaitCondition'\n",
                "$region = '",
                { "Ref": "AWS::Region" },
                "'\n$stack = '",
                { "Ref": "AWS::StackId" },
                "'\n$output = (Get-CFNStackResources -StackName $stack -LogicalResourceId $resource -Region $region)\n",
                "while (($output -eq $null) -or ($output.ResourceStatus -ne 'CREATE_COMPLETE') -and ($output.ResourceStatus -ne 'UPDATE_COMPLETE')) {\n",
                "    Start-Sleep 5\n",
                "    $output = (Get-CFNStackResources -StackName $stack -LogicalResourceId $resource -Region $region)\n",
                "}\n",
                "### Join the domain...\n",             
                "$computer = Get-WmiObject -Class Win32_ComputerSystem \n",
                "if ($computer.domain -eq 'WORKGROUP') { \n",
                "  $adapter = Get-NetAdapter -Name 'Ethernet*'\n",
                "  Set-DNSClientServerAddress -InterfaceAlias $adapter.Name -ServerAddresses ('",
                {
                  "Fn::GetAtt": [
                    "DC1",
                    "PrivateIp"
                  ]
                },
                "')\n",
                "  $domain = '",
                {
                  "Ref": "DomainDNSName"
                },
                "'\n",
                "  $password = '",
                {
                  "Ref": "AdminPassword"
                },
                "' | ConvertTo-SecureString -asPlainText -Force \n",
                "  $Administrator = '",
                {
                  "Ref": "DomainNetBiosName"
                },
                "\\",
                {
                  "Ref": "DomainAdminUser"
                },
                "'\n",
                "  $credential = New-Object System.Management.Automation.PSCredential($Administrator,$password) \n",
                "  Add-Computer -DomainName $domain -Credential $credential -restart \n",
                "}\n",
                "</powershell>\n"
              ]
            ]
          }
        }
      },
      "Type": "AWS::EC2::Instance"
    },
    
    "PrivateSecurityGroup": {
      "Properties": {
        "GroupDescription": "This is the security group for Active Directory",
        "SecurityGroupEgress": [
        
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "tcp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "udp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          },
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "80",
            "IpProtocol": "tcp",
            "ToPort": "80"
          },
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "443",
            "IpProtocol": "tcp",
            "ToPort": "443"
          }
        ],
        "SecurityGroupIngress": [
          {
            "FromPort": "3389",
            "IpProtocol": "tcp",
            "SourceSecurityGroupId": {
              "Ref": "BastionSecurityGroup"
            },
            "ToPort": "3389"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "tcp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "udp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "PrivateSecurityGroup"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::SecurityGroup"
    },
    "BastionSecurityGroup": {
      "Properties": {
        "GroupDescription": "This is the security group for the bastion host",
        "SecurityGroupEgress": [
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "80",
            "IpProtocol": "tcp",
            "ToPort": "80"
          },
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "443",
            "IpProtocol": "tcp",
            "ToPort": "443"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "3389",
            "IpProtocol": "tcp",
            "ToPort": "3389"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "tcp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "0",
            "IpProtocol": "udp",
            "ToPort": "65535"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          }
        ],
        "SecurityGroupIngress": [
          {
            "CidrIp": "0.0.0.0/0",
            "FromPort": "3389",
            "IpProtocol": "tcp",
            "ToPort": "3389"
          },
          {
            "CidrIp": "10.1.0.0/16",
            "FromPort": "-1",
            "IpProtocol": "icmp",
            "ToPort": "-1"
          }
        ],
        "Tags": [
          {
            "Key": "Name",
            "Value": "BastionSecurityGroup"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::SecurityGroup"
    },

    "PublicSubnetAZ1": {
      "Properties": {
        "AvailabilityZone": {
          "Fn::Select": [
            0,
            {
              "Fn::GetAZs": ""
            }
          ]
        },
        "CidrBlock": "10.1.1.0/24",
        "Tags": [
          {
            "Key": "Name",
            "Value": "PublicSubnetAZ1"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::Subnet"
    },
    "PublicSubnetAZ1SubnetAssociation": {
      "Properties": {
        "RouteTableId": {
          "Ref": "DMZRouteTable"
        },
        "SubnetId": {
          "Ref": "PublicSubnetAZ1"
        }
      },
      "Type": "AWS::EC2::SubnetRouteTableAssociation"
    },

    "PrivateSubnetAZ1": {
      "Properties": {
        "AvailabilityZone": {
          "Fn::Select": [
            0,
            {
              "Fn::GetAZs": ""
            }
          ]
        },
        "CidrBlock": "10.1.3.0/24",
        "Tags": [
          {
            "Key": "Name",
            "Value": "PrivateSubnetAZ1"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::Subnet"
    },
    "PrivateSubnetAZ1SubnetAssociation": {
      "Properties": {
        "RouteTableId": {
          "Ref": "InternalRouteTable"
        },
        "SubnetId": {
          "Ref": "PrivateSubnetAZ1"
        }
      },
      "Type": "AWS::EC2::SubnetRouteTableAssociation"
    },
    
    "DMZRouteTable": {
      "Properties": {
        "Tags": [
          {
            "Key": "Application",
            "Value": {
              "Ref": "AWS::StackName"
            }
          },
          {
            "Key": "Name",
            "Value": "DMZRouteTable"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::RouteTable"
    },
    "InternalRouteTable": {
      "Properties": {
        "Tags": [
          {
            "Key": "Name",
            "Value": "InternalRouteTable"
          }
        ],
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::RouteTable"
    },
    
    "NATRoute": {
      "Properties": {
        "DestinationCidrBlock": "0.0.0.0/0",
        "NatGatewayId": {
          "Ref": "AWSNat"
        },
        "RouteTableId": {
          "Ref": "InternalRouteTable"
        }
      },
      "Type": "AWS::EC2::Route"
    },
    "PublicRoute": {
      "DependsOn" : "InternetGatewayAttachment",
      "Properties": {
        "DestinationCidrBlock": "0.0.0.0/0",
        "GatewayId": {
          "Ref": "InternetGateway"
        },
        "RouteTableId": {
          "Ref": "DMZRouteTable"
        }
      },
      "Type": "AWS::EC2::Route"
    },
    "VPC": {
      "Properties": {
        "CidrBlock": "10.1.0.0/16",
        "EnableDnsHostnames": "true",
        "EnableDnsSupport": "true",
        "Tags": [
          {
            "Key": "Name",
            "Value": "VPC"
          }
        ]
      },
      "Type": "AWS::EC2::VPC"
    },
    "InternetGateway": {
      "Properties": {
        "Tags": [
          {
            "Key": "Name",
            "Value": "InternetGateway"
          }
        ]
      },
      "Type": "AWS::EC2::InternetGateway"
    },
    "InternetGatewayAttachment": {
      "Properties": {
        "InternetGatewayId": {
          "Ref": "InternetGateway"
        },
        "VpcId": {
          "Ref": "VPC"
        }
      },
      "Type": "AWS::EC2::VPCGatewayAttachment"
    },
    "AWSNat": {
      "DependsOn": "InternetGatewayAttachment", 
      "Properties": {
        "AllocationId": {
          "Fn::GetAtt": [
            "NatEip",
            "AllocationId"
          ]
        },
        "SubnetId": {
          "Ref": "PublicSubnetAZ1"
        }
      },
      "Type": "AWS::EC2::NatGateway"
    },
    "NatEip": {
      "Properties": {
        "Domain": "vpc"
      },
      "Type": "AWS::EC2::EIP"
    }
  }
}

Alas, there is a drawback to this approach. Sysprep is not supported for the Active Directory role, so you won’t be able to have more than one domain controller built from your custom AMI in the same VPC. But there’s a workaround. If you are building a production SharePoint farm spanning two Availability Zones, you could still use this technique to optimize the launch time of the first domain controller. You would then build the other domain controllers as Domain-Join2.json does, and join all of the application servers and other domain controllers to the forest after it’s ready (as done in Domain-Join3.json). Although it sounds like more work, this approach allows you to launch all of the instances simultaneously.

Other best practices

Here are a couple tips to help you diagnose CloudFormation stack failures.

Start by examining the Events tab in the CloudFormation console, of course. But sometimes the stack appears to succeed even if there is an error in your Windows initialization logic. For that, you need to remotely connect to the instance and study these two log files:

  • C:\Program Files\Amazon\Ec2ConfigService\Logs\Ec2ConfigLog.txt
  • C:\cfn\log\cfn-init.log

These logs aren’t formatted very well in Notepad on Windows Server, so I copy and paste the whole file into WordPad on my local workstation. That also allows me to study the error messages even if the instance terminates.

Sometimes the stack fails in CloudFormation, but only because it timed out waiting for the PowerShell initiation logic to complete. In that case, you need to set the CloudFormation Rollback on failure setting to No , in the Advanced options section for your stack.

Conclusion

This post described some best practices for working with CloudFormation and a few ways to join Windows Server instances to a domain. Using PowerShell, I reduced the time to create a stack from about 20 minutes to about 7 minutes.

In addition to these methods of running your own domain controller in Amazon EC2, another option is to use the AWS managed service, AWS Directory Service. It makes connecting instances to a domain easy. For more information, see How to Configure Your EC2 Instances to Automatically Join a Microsoft Active Directory Domain.

If you have questions or suggestions, please comment below.