Category: AWS Tools for PowerShell*

AWS Support for PowerShell Core 6.0

Announced in a Microsoft blog post yesterday, PowerShell Core 6.0 is now generally available. AWS continues to support this new cross-platform version of PowerShell with our AWS Tools for PowerShell Core module also known by its module name, AWSPowerShell.NetCore. This post recaps the modules available from AWS for PowerShell users wanting to script their AWS resources.

AWS Tools for Windows PowerShell

Released in 2012, this module, also known by the module name AWSPowerShell, supports users working with the traditional Windows-only version of PowerShell. It supports PowerShell version 2.0 through to 5.1. It can be installed from the AWS SDK and Tools for .NET Windows Installer, which also contains .NET 3.5 and 4.5 versions of the AWS SDK for .NET and the AWS Toolkit for Visual Studio 2013 and 2015. The module is also distributed on the PowerShell Gallery and is pre-installed on Amazon EC2 Windows-based images.

AWS Tools for PowerShell Core

This version of the tools was first released in August 2016 to coincide with the announcement of the first public alpha release of PowerShell Core 6.0. Since then it has continued to be updated in sync with the AWS Tools for Windows PowerShell module. This module, named AWSPowerShell.NetCore, is only distributed on the PowerShell Gallery.

Module Compatibility

Both modules are highly compatible with each other. In terms of the cmdlets they expose for AWS service APIs, they match completely and both modules are updated in sync. As noted in our original launch blog post for our module running on PowerShell Core, back in August 2016, the AWSPowerShell.NetCore module is missing only a handful of cmdlets, as follows.

Proxy cmdlets:


Logging cmdlets:


SAML federated credentials cmdlets:


Now that PowerShell Core is generally available (GA), we’ll be taking another look at these to see if we can add them.

We hope you’re enjoying the new PowerShell Core GA release and the ability to script and access your AWS resources from PowerShell on any system!

Send Real-Time Amazon CloudWatch Alarm Notifications to Amazon Chime

This post was authored by Trevor Sullivan, a Solutions Architect for Amazon Web Services (AWS) based in Seattle, Washington. The post was also peer-reviewed by Andy Elmhorst, Senior Solutions Architect for AWS.


When you’re developing, deploying, and supporting business-critical applications, timely system notifications are crucial to keeping your services up and running reliably for your customers. If your team actively collaborates using Amazon Chime, you might want to receive critical system notifications directly within your team chat rooms. This is possible using the Amazon Chime incoming webhooks feature.

Using Amazon CloudWatch alarms, you can set up metric thresholds and send alerts to Amazon Simple Notification Service (SNS). SNS can send notifications using e-mail, HTTP(S) endpoints, and Short Message Service (SMS) messages to mobile phones, and it can even trigger a Lambda function.

Because SNS doesn’t currently support sending messages directly to Amazon Chime chat rooms, we’ll insert a Lambda function in between them. By triggering a Lambda function from SNS instead, we can consume the event data from the CloudWatch alarm and craft a human-friendly message before sending it to Amazon Chime.

Here’s a simple architectural diagram that demonstrates how the various components will work together to make this solution work. Feel free to refer back to this diagram as you make your way through the remainder of this article.


Throughout this article, we make the following assumptions:

  • You have created an Amazon EC2 instance running Ubuntu Linux.
  • Detailed CloudWatch monitoring is enabled for this EC2 instance.
  • Amazon Chime is already set up and accessible to you.
  • You’ve installed PowerShell Core, or can run it in a Docker container.
  • You have installed and configured IAM credentials for the AWS Tools for PowerShell.
  • Python 3.6 and pip3 are installed on your development system.

NOTE: There is an additional cost to capture detailed CloudWatch metrics for EC2 instances, detailed here.

Set up Amazon Chime

Before implementing your backend application code, there are a couple steps you need to perform within Amazon Chime. To set up your incoming webhook, you first need to create a new Amazon Chime chat room. Webhooks are created as a resource in the context of the chat room. As of this writing, Chime Webhooks must be created using the native Amazon Chime client for Microsoft Windows or Apple MacOS.

Create an Amazon Chime chat room

First you create a new chat room in Amazon Chime. You’ll use this chat room for testing and, once you understand and successfully implement this solution, you can replicate it in your live chat rooms.

  1. Open Amazon Chime.
  2. Choose the Rooms button.
  3. Choose the New room button.
  4. Give the chat room a name, and then choose the Create button.

Create an Amazon Chime incoming webhook

Now that you’ve created your new Amazon Chime chat room, you need to generate a webhook URL. This webhook URL authorizes your application to send messages to the chat room. Be sure to handle the URL with the same level of security that you would handle any other secrets or passwords.

In the Amazon Chime chat room, click the gear icon, and then select the Manage Webhooks menu item. In the webhook management window, choose the New button and use the name CriticalAlerts. Click the Copy webhook URL link and paste it into a temporary notepad. We’ll need to configure this URL on our Lambda function later on.

Create an SNS topic

In this section, you create a Simple Notification Service (SNS) topic. This SNS topic will be triggered by a CloudWatch alarm when its configured metric threshold is exceeded. You can name the SNS topic whatever you prefer, but in this example, I’ll use the name chimewebhook.

It’s possible to create the SNS topic after creating your CloudWatch alarm. However, in this case, you would have to go back and reconfigure your CloudWatch alarm to point to the new SNS topic. In this example, we create the topic first to minimize the amount of context switching between services.

Use the following PowerShell command to create an SNS topic, and store the resulting topic Amazon Resource Name (ARN) in a variable named $Topic. We’ll use this variable later on, so don’t close your PowerShell session.

$TopicArn = New-SNSTopic -Name chimewebhook -Region us-west-2

Create a CloudWatch alarm

In this section, you create an Amazon CloudWatch alarm. Then you configure this alarm to trigger an alert state when the CPU usage metric of your EC2 instance exceeds 10%. Alarms can be configured with zero or more actions; you’ll configure a single action to send a notification to the SNS topic you previously created.

  • Navigate to the CloudWatch alarms feature in the AWS Management Console.
  • Choose the blue Create Alarm button.
  • Search for your EC2 instance ID.
  • Select the CPUUtilization metric for your EC2 instance.
  • On the next screen, give the CloudWatch alarm a name and useful description.
  • Configure the CPUUtilization threshold for 10%, and be sure the Period is set to 1 minute.
  • In the Actions section, select your SNS topic.
  • Save the CloudWatch alarm.

If you’d prefer to use a PowerShell script to deploy the CloudWatch alarm, use the following example script. Be sure you specify the correct parameter values for your environment:

  • EC2 instance ID that you’re monitoring
  • AWS Region that your EC2 instance resides in
  • ARN of the SNS topic that CloudWatch will publish alerts to
### Create a CloudWatch dimension object, to alarm against the correct EC2 instance ID
$MetricDimension = [Amazon.CloudWatch.Model.Dimension]::new()
$MetricDimension.Name = 'InstanceId'
$MetricDimension.Value = 'i-04043befbbfcdc51e'

### Set up the parameters to create a CloudWatch alarm in a PowerShell HashTable
$Alarm = @{
  AlarmName = 'EC2 instance exceeded 10% CPU'
  ActionsEnabled = $true
  AlarmAction = $TopicArn
  ComparisonOperator = ([Amazon.CloudWatch.ComparisonOperator]::GreaterThanOrEqualToThreshold)
  Threshold = 10
  Namespace = 'AWS/EC2'
  MetricName = 'CPUUtilization'
  Dimension = $MetricDimension
  Period = 60
  EvaluationPeriod = 1
  Statistic = [Amazon.CloudWatch.Statistic]::Maximum
  Region = 'us-west-2'
Write-CWMetricAlarm @Alarm

Set up AWS Lambda

In this section, you will create an AWS Lambda function, based on Python 3, that will be triggered by the SNS topic that you created earlier. This Lambda function will parse some of the fields of the message that’s forwarded from CloudWatch to SNS.

Create the Lambda Function

To successfully invoke an Amazon Chime webhook, your HTTP invocation must follow these criteria:

  • Webhook URL is predefined by Amazon Chime
  • Request is sent using the HTTP POST verb
  • Content-Type HTTP header must be application/json
  • HTTP body must contain a JSON object with Content property

We’ll use the open source Python requests library to make the Amazon Chime webhook invocation, as it provides a simple development interface. Because you’re adding a dependency on an external library, you need to author your Lambda function locally, package it up into a ZIP archive, and then deploy the ZIP archive to Lambda.

Start by creating the following three files in a working directory.

This file contains the AWS Lambda function that is invoked when a CloudWatch alarm is triggered.

import os
import boto3
import requests
from base64 import b64decode

def get_message(event):
  This function retrieves the message that will be sent to the Amazon Chime webhook. If the Lambda
  function is triggered manually, it will return some static text. However, if the Lambda function
  is invoked by SNS from CloudWatch Alarms, it will emit the Alarm's subject line.
    return event['Records'][0]['Sns']['Subject']
  except KeyError:
    return 'test message'

def handler(event, context):
  The 'handler' Python function is the entry point for AWS Lambda function invocations.
  print('Getting ready to send message to Amazon Chime room')
  content = 'CloudWatch Alarm! {0}'.format(get_message(event))
  webhook_uri = os.environ['CHIME_WEBHOOK'], json={ 'Content': content })
  print('Finished sending notification to Amazon Chime room')





Build and deploy the Lambda package

Now that you’ve created the previous source files, you’ll need a PowerShell script to build the ZIP archive for Lambda, create the Lambda function, and give SNS access to invoke the Lambda function. Save the following PowerShell script file into the same working directory, update the <YourChimeWebhookURL> text with your actual Amazon Chime webhook URL, and then run the script.

NOTE: This PowerShell script has a dependency on the Mac and Linux zip utility. If you’re running this code on Windows 10, you can use the Compress-Archive PowerShell command, or run the PowerShell script in the Windows Subsystem for Linux feature.


Set-DefaultAWSRegion -Region us-west-2

$ZipFileName = ''

Set-Location -Path $PSScriptRoot

Write-Host -Object 'Restoring dependencies ...'
pip3 install -r $PSScriptRoot/requirements.txt -t $PSScriptRoot/

Write-Host -Object 'Compressing files ...'
Get-ChildItem -Recurse | ForEach-Object -Process {
  $NewPath = $PSItem.FullName.Substring($PSScriptRoot.Length + 1)
  zip -u "$PSScriptRoot/$ZipFileName" $NewPath

Write-Host -Object 'Deploying Lambda function'

$Function = @{
  FunctionName = 'AmazonChimeAlarm'
  Runtime = 'python3.6'
  Description = 'Sends a message to an Amazon Chime room when a CloudWatch alarm is triggered.'
  ZipFilename = $ZipFileName
  Handler = 'index.handler'
  Role = 'arn:aws:iam::{0}:role/service-role/lambda' -f (Get-STSCallerIdentity).Account
  Environment_Variable = @{
    CHIME_WEBHOOK = '<YourChimeWebhookURL>'
Remove-LMFunction -FunctionName $Function.FunctionName -Force
Publish-LMFunction @Function

Write-Host -Object 'Deployment completed' -ForegroundColor Green

Once you’ve executed this deployment script, you should see an AWS Lambda function named AmazonChimeAlarm in the AWS Management Console.

Configure Lambda function policy

AWS Lambda functions have their own resource-level policies that are somewhat similar to IAM policies. These function policies are what grant other cloud resources the access that they need to invoke the function. In this scenario, you need to grant the SNS service access to trigger your Lambda function.

The following PowerShell script adds the necessary permissions to your Lamdba function.

### Enables the Amazon Simple Notification Service (SNS) to invoke your Lambda function
$LMPermission = @{
  FunctionName = $Function.FunctionName
  Action = 'lambda:InvokeFunction'
  Principal = ''
  StatementId = 1
Add-LMPermission @LMPermission

Keep in mind that this Lambda function policy broadly allows any SNS topic, in any AWS account, to trigger your Lambda function. For production applications, you should use the -SourceArn parameter to restrict access to specific event sources that will be able to trigger your Lambda function.

Subscribe the Lambda function to the SNS topic

Now you’ve created your Lambda function, and granted access to SNS to trigger it, you need to subscribe the Lambda function to the topic. This subscription/association is what starts the flow of events from SNS to Lambda. Without the subscription, CloudWatch alarms would trigger your SNS topic successfully, but the event flow would stop there.

$Subscription = @{
  Protocol = 'lambda'
  Endpoint = $NewFunction.FunctionArn
  TopicArn = $TopicArn
Connect-SNSNotification @Subscription

Trigger a test event

Now that you’ve finished configuring your AWS account, you can go ahead and test the end-to-end process to ensure it’s working properly. Ensure you’ve got your Amazon Chime client running, and select your test chat room that you created earlier.

Next, invoke a process on your instance that will consume many CPU cycles. Connect to your EC2 instance using SSH and run the following shell commands.

sudo apt install --yes
sudo usermod --append --groups docker ubuntu
exit # SSH back in after this, so group memberships take effect

docker run --rm --detach trevorsullivan/cpuburn burnP5

This Ubuntu-based Docker container image contains the preinstalled CPU burn program, which will cause your EC2 instance’s CPU consumption to spike to 100%. Because you’ve enabled detailed CloudWatch metrics on your EC2 instance, after a couple of minutes, the CloudWatch alarm that you created should get triggered.

Once you’ve finished with the test, or if you want to trigger the CloudWatch alarm again, make sure that you stop the Docker container running the CPU burn program. Because you specified the --rm argument upon running the container, the container will be deleted after it has stopped.

docker ps # Find the container ID
docker rm -f <containerID> # Remove the container

Potential problems

If you run into any problems with testing the end-to-end solution, check out the following list of potential issues you could run into and ways to troubleshoot:

  • The CPU burn program might not result in adequate CPU consumption, which wouldn’t trigger the test event correctly. Use the Linux top command to ensure that you trigger the test event. Or simply pull up the CPUUtilization metric in CloudWatch and see what values are being recorded.
  • If your Lambda function is not correctly configured to accept invocations from SNS, your SNS topic will fail to invoke it. Be sure that you understand how Lambda function policies work, and ensure that your Lambda function has the appropriate resource-level IAM policy to enable SNS to invoke it.
  • By default, your EC2 instances include basic metrics for a five-minute period. If you don’t have detailed monitoring enabled for the EC2 instance you’ve used in this article, you might have to wait several minutes for the next metric data point to be recorded. For more immediate feedback, ensure that your EC2 instance has detailed monitoring configured, so that you get instance-level metrics on a per-minute period instead.
  • Ensure your Lambda function is subscribed to your SNS topic. If your SNS topic doesn’t have any subscribers, it won’t know how to “handle” the alarm state from CloudWatch, and will effectively discard the message.
  • If you aren’t receiving any notifications in your Amazon Chime chat room, ensure that your CloudWatch alarm is in an OK state before attempting to retrigger it. CloudWatch sends a single notification to the configured SNS topics when the alarm’s threshold is breached, and doesn’t continually send notifications while it’s in an alarm state.
  • If your HTTP POST request to the Chime Webhook URL fails with HTTP 429, then your application might be rate-limited by Amazon Chime. Please refer to the official product documentation for more information. As of this writing, Chime Webhook URLs support 1 transaction per second (TPS).


In this article, you configured your AWS account to send Amazon Chime notifications to a team chat room, when CloudWatch alarm thresholds are breached. You can repeat this process as many times as you need to, in order to send the right notifications to your team’s chat room. Chat rooms in general can get noisy quickly, if you don’t take care to determine which notifications are the most critical to your team. Hence, I recommend that you discuss with your team which notifications you need immediate notification about, before spending the effort to build a solution using this technique.

Thanks for taking the time to read this article and learn about how you can integrate Amazon Chime with your critical system notifications.

Build on!
Trevor Sullivan, Solutions Architect
Amazon Web Services (AWS)


Deploy an Amazon ECS Cluster Running Windows Server with AWS Tools for PowerShell – Part 1

This is a guest post from Trevor Sullivan, a Seattle-based Solutions Architect at Amazon Web Services (AWS). In this blog post, Trevor shows you how to deploy a Windows Server-based container cluster using the AWS Tools for PowerShell.

Building and deploying applications on the Windows Server platform is becoming a significantly lighter-weight process. Although you might be accustomed to developing monolithic applications on Windows, and scaling vertically instead of horizontally, you might want to rethink how you design and build Windows-based apps. Now that application containers are a native feature in the Windows Server platform, you can design your applications in a similar fashion to how developers targeting the Linux platform have designed them for years.

Throughout this blog post, we explore how to automate the deployment of a Windows Server container cluster, managed by the Amazon EC2 Container Service (Amazon ECS). The architecture of the cluster can help you efficiently and cost-effectively scale your containerized application components.

Due to the large amount of information contained in this blog post, we’ve separated it into two parts. Part 1 guides you through the process of deploying an ECS cluster. Part 2 covers the creation and deployment of your own custom container images to the cluster.


In this blog post, we assume the following:

  • You’re using a Windows, Mac, or Linux system with PowerShell Core installed and you’ve installed the AWS Tools for PowerShell Core.
  • Or you’re using a Windows 7 or later system, with Windows PowerShell 4.0 or later installed (use $PSVersionTable to check your PowerShell version) and have installed the AWS Tools for PowerShell (aka AWSPowerShell module).
  • You already have access to an AWS account.
  • You’ve created an AWS Identity and Access Management (IAM) user account with administrative policy permissions, and generated an access key ID and secret key.
  • You’ve configured your AWS access key ID and secret key in your ~/.aws/credentials file.

This article was authored, and the code tested, on a MacBook Pro with PowerShell Core Edition and the AWSPowerShell.NetCore module.

NOTE: In the PowerShell code examples, I’m using a technique known as PowerShell Splatting to pass parameters into various PowerShell commands. Splatting syntax helps your PowerShell code look cleaner, enables auditing of parameter values prior to command invocation, and improves code maintenance.

Create your Amazon ECS cluster

The first task is to create an empty ECS cluster. Right now, the ECS beta support for Windows Server containers doesn’t enable you to provision your compute layer for the cluster, at cluster creation time. In a moment, you’ll provision some Windows Server-based compute capacity on Amazon EC2, and associate it with your empty ECS cluster.

If you prefer to use the AWS PowerShell module to create the empty ECS cluster, you can use the following command.

New-ECSCluster -ClusterName ECSRivendell

Set up an IAM role for Amazon EC2 instances

Amazon EC2 instances can have IAM “roles” assigned to them. By associating one or more IAM policies with an IAM role, which is assigned to an EC2 instance by way of an “instance profile”, you can grant access to various services in AWS directly to the EC2 instance, without having to embed and manage any credentials. AWS handles that for you, via the IAM role. Whenever the EC2 instance is running, it has access to the IAM role that’s assigned to it, and can make calls to various AWS APIs that it’s authorized to call via associated IAM policies.

Before you actually deploy the Windows Server compute layer for your ECS cluster, you must first perform the following preparation steps:

  • Create an IAM policy that defines the required permissions for ECS container instances.
  • Create an IAM role.
  • Register the IAM policy with the role.
  • Create an IAM instance profile, which will be associated with your EC2 instances.
  • Associate the IAM role with the instance profile.

Create an IAM policy for your EC2 container instances

First, you use PowerShell to create the IAM policy. I simply copied and pasted the IAM policy that’s documented in the Amazon ECS documentation. This IAM policy will be associated with the IAM role that will be associated with your EC2 container instances, which actually run your containers via ECS. This policy is especially important, because it grants your container instances access to Amazon EC2 Container Registry (Amazon ECR). This is a private storage area for your container images, similar to the Docker Store.

$IAMPolicy = @{
    Path = '/ECSRivendell/'
    PolicyName = 'ECSRivendellInstanceRole'
    Description = 'This policy is used to grant EC2 instances access to ECS-related API calls. It enables EC2 instances to push and pull from ECR, and most importantly, register with our ECS Cluster.'
    PolicyDocument = @'
    "Version": "2012-10-17",
    "Statement": [
        "Effect": "Allow",
        "Action": [
        "Resource": "*"
New-IAMPolicy @IAMPolicy

Create a container instance IAM role

The next part is easy. You just need to create an IAM role. This role has a name, optional path, optional description and, the important part, an “AssumeRolePolicyDocument”. This is also known as the IAM “trust policy”. This IAM trust policy enables the EC2 instances to use or “assume” the IAM role, and use its policies to access AWS APIs. Without this trust policy in place, your EC2 instances won’t be able to use this role and the permissions granted to it by the IAM policy.

$IAMRole = @{
    RoleName = 'ECSRivendell'
    Path = '/ECSRivendell/'
    Description = 'This IAM role grants the container instances that are part of the ECSRivendell ECS cluster access to various AWS services, required to operate the ECS Cluster.'
    AssumeRolePolicyDocument = @'
    "Version": "2008-10-17",
    "Statement": [
        "Sid": "",
        "Effect": "Allow",
        "Principal": {
            "Service": ""
        "Action": "sts:AssumeRole"
New-IAMRole @IAMRole

Register the IAM policy with the IAM role

Now you simply need to associate the IAM policy that you created with the IAM role. This association is very easy to make with PowerShell, using the Register-IAMRolePolicy command. If necessary, you can associate more than one policy with your IAM roles that grant or deny additional permissions.

$RolePolicy = @{
    PolicyArn = $NewIAMPolicy.Arn
    Role = $NewIAMRole.RoleName
Register-IAMRolePolicy @RolePolicy

Create an IAM instance profile

With the IAM role prepared, you now need to create the IAM instance profile. The IAM instance profile is the actual object that gets associated with our EC2 instances. This IAM instance profile ultimately grants the EC2 instance permission to call AWS APIs directly, without any stored credentials.

$InstanceProfile = @{
    InstanceProfileName = 'ECSRivendellInstanceRole'
    Path = '/ECSRivendell/'
$NewInstanceProfile = New-IAMInstanceProfile @InstanceProfile

Associate the IAM role with the instance profile

Finally, you need to associate our IAM Role with the instance profile.

$RoleInstanceProfile = @{
    InstanceProfileName = $NewInstanceProfile.InstanceProfileName
    RoleName = $NewIAMRole.RoleName
$null = Add-IAMRoleToInstanceProfile @RoleInstanceProfile

Add EC2 compute capacity for Windows

Now that you’ve created an empty ECS cluster and have pre-staged your IAM configuration, you need to add some EC2 compute instances running Windows Server to it. These are the actual Windows Server instances that will run containers (tasks) via the ECS cluster scheduler.

To provision EC2 capacity in a frugal and scalable fashion, we create an EC2 Auto Scaling group using Spot instances. Spot instances are one of my favorite services in AWS, because you can provision a significant amount of compute capacity for a huge discount, by bidding on unused capacity. There is some risk that you might need to design around, as Spot instances can be terminated or stopped if your bid price is exceeded. However, depending on your needs, you can run powerful applications on EC2, for a low price, using EC2 Spot instances.

Create the Auto Scaling launch configuration

Before you can create the Auto Scaling group itself, you need to create what’s known as an Auto Scaling “launch configuration”. The launch configuration is essentially a blueprint for Auto Scaling groups. It stores a variety of input parameters that define how new EC2 instances will look every time the Auto Scaling group scales up, and spins up a new instance. For example, you need to specify:

  • The Amazon Machine Image (AMI) that instances will be launched from.
    • NOTE: Be sure you use the Microsoft Windows Server 2016 Base with Containers image for Windows Server container instances.
  • The EC2 instance type (size, vCPUs, memory).
  • Whether to enable or disable public IP addresses for EC2 instances.
  • The bid price for Spot instances (optional, but recommended).
  • The EC2 User Data – PowerShell script to bootstrap instance configuration.

Although the code snippet below might look a little scary, it basically contains an EC2 user data script, which I copied directly from the AWS beta documentation for Windows. This automatically installs the ECS container agent into new instances that are brought online by your Auto Scaling Group, and registers them with your ECS cluster.

$ClusterName = 'ECSRivendell'
$UserDataScript = @'

## The string 'windows' should be replaced with your cluster name

# Set agent env variables for the Machine context (durable)
[Environment]::SetEnvironmentVariable("ECS_CLUSTER", "YOURCLUSTERNAME", "Machine")
[Environment]::SetEnvironmentVariable("ECS_ENABLE_TASK_IAM_ROLE", "true", "Machine")
$agentVersion = 'v1.14.5'
$agentZipUri = "$"
$agentZipMD5Uri = "$agentZipUri.md5"

### --- Nothing user configurable after this point ---
$ecsExeDir = "$env:ProgramFiles\Amazon\ECS"
$zipFile = "$env:TEMP\"
$md5File = "$env:TEMP\"

### Get the files from Amazon S3
Invoke-RestMethod -OutFile $zipFile -Uri $agentZipUri
Invoke-RestMethod -OutFile $md5File -Uri $agentZipMD5Uri

## MD5 Checksum
$expectedMD5 = (Get-Content $md5File)
$md5 = New-Object -TypeName System.Security.Cryptography.MD5CryptoServiceProvider
$actualMD5 = [System.BitConverter]::ToString($md5.ComputeHash([System.IO.File]::ReadAllBytes($zipFile))).replace('-', '')

if($expectedMD5 -ne $actualMD5) {
    echo "Download doesn't match hash."
    echo "Expected: $expectedMD5 - Got: $actualMD5"
    exit 1

## Put the executables in the executable directory
Expand-Archive -Path $zipFile -DestinationPath $ecsExeDir -Force

## Start the agent script in the background
$jobname = "ECS-Agent-Init"
$script =  "cd '$ecsExeDir'; .\amazon-ecs-agent.ps1"
$repeat = (New-TimeSpan -Minutes 1)

$jobpath = $env:LOCALAPPDATA + "\Microsoft\Windows\PowerShell\ScheduledJobs\$jobname\ScheduledJobDefinition.xml"
if($(Test-Path -Path $jobpath)) {
    echo "Job definition already present"
    exit 0


$scriptblock = [scriptblock]::Create("$script")
$trigger = New-JobTrigger -At (Get-Date).Date -RepeatIndefinitely -RepetitionInterval $repeat -Once
$options = New-ScheduledJobOption -RunElevated -ContinueIfGoingOnBattery -StartIfOnBattery
Register-ScheduledJob -Name $jobname -ScriptBlock $scriptblock -Trigger $trigger -ScheduledJobOption $options -RunNow
Add-JobTrigger -Name $jobname -Trigger (New-JobTrigger -AtStartup -RandomDelay 00:1:00)

'@ -replace 'YOURCLUSTERNAME', $ClusterName

$UserDataBase64 = [System.Convert]::ToBase64String(([Byte[]][Char[]] $UserDataScript))

# Create a block device mapping to increase the size of the root volume
$BlockDevice = [Amazon.AutoScaling.Model.BlockDeviceMapping]::new()
$BlockDevice.DeviceName = '/dev/sda1'
$BlockDevice.Ebs = [Amazon.AutoScaling.Model.Ebs]::new()
$BlockDevice.Ebs.DeleteOnTermination = $true
$BlockDevice.Ebs.VolumeSize = 200
$BlockDevice.Ebs.VolumeType = 'gp2'

$LaunchConfig = @{
    LaunchConfigurationName = 'ECSRivendell'
    AssociatePublicIpAddress = $true
    EbsOptimized = $true
    BlockDeviceMapping = $BlockDevice
    InstanceType = 'r4.large'
    SpotPrice = '0.18'
    InstanceMonitoring_Enabled = $true
    IamInstanceProfile = 'ECSRivendellInstanceRole'
    ImageId = 'ami-6a887b12'
    UserData = $UserDataBase64
$NewLaunchConfig = New-ASLaunchConfiguration @LaunchConfig

Be sure you give your EC2 container instances enough storage to cache container images over time. In my example, I’m instructing the launch configuration to grant a 200 GB Amazon EBS root volume to each EC2 instance that is deployed into the Auto Scaling group, once it’s set up.

Create the Auto Scaling group

Now that you’ve set up the Auto Scaling launch configuration, you can deploy a new Auto Scaling group from it. Once created, the Auto Scaling group actually submits the EC2 Spot request for the desired numbers of EC2 instances when they’re needed. All you have to do is tell the Auto Scaling group how many instances you want, and it handles spinning them up or down.

Optionally, you can even set up Auto Scaling policies so that you don’t have to manually scale the cluster. We’ll save the topic of Auto Scaling policies for another article, however, and instead focus on setting up the ECS cluster.

The PowerShell code to create the Auto Scaling group, from the launch configuration that you built, is included below. Keep in mind that although I’ve tried to keep the code snippets fairly generic, this particular code snippet does have a hard-coded command that retrieves all of your Amazon Virtual Private Cloud (VPC) subnet IDs dynamically. It should work fine, as long as you only have a single VPC in the region you’re currently operating in. If you have more than one VPC, you’ll want to filter out the list of subnets from the Get-EC2Subnet command that are contained in the desired target VPC.

$AutoScalingGroup = @{
    AutoScalingGroupName = 'ECSRivendell'
    LaunchConfigurationName = 'ECSRivendell'
    MinSize = 2  
    MaxSize = 5
    DesiredCapacity = 3
    VPCZoneIdentifier = [String]::Join(',', (Get-EC2Subnet).SubnetId)
    Tag = $( $Tag = [Amazon.AutoScaling.Model.Tag]::new()
             $Tag.PropagateAtLaunch = $true; $Tag.Key = 'Name'; $Tag.Value = 'ECS: ECSRivendell'; $Tag )
$NewAutoScalingGroup = New-ASAutoScalingGroup @AutoScalingGroup

After running this code snippet, your Auto Scaling Group will take a few minutes to deploy. Additionally, it can sometimes take 15-30 minutes for the ECS container agent, running on each of the EC2 instances, to finish registering with the ECS cluster. Once you’ve waited a little while, check out the EC2 instance area of the AWS Management Console and examine your shiny new container instances!

You can also visit the ECS area of the AWS Management Console, and examine the number of container instances that are registered with the ECS cluster. You now have a functioning ECS cluster!

You can even use the Get-ECSClusterDetail command in PowerShell to examine the current count of container instances.

PS /Users/tsulli> (Get-ECSClusterDetail -Cluster ECSRivendell).Clusters

ActiveServicesCount               : 0
ClusterArn                        : arn:aws:ecs:us-west-2:676655494252:cluster/ECSRivendell
ClusterName                       : ECSRivendell
PendingTasksCount                 : 0
RegisteredContainerInstancesCount : 3
RunningTasksCount                 : 0
Status                            : ACTIVE


In this article, you set up an EC2 Container Service (ECS) cluster with several EC2 instances running Windows Server registered to it. Prior to that, you also set up the IAM policy, IAM role, and instance profile that are necessary prerequisites to ensure that the ECS cluster operates correctly. You used the AWS Tools for PowerShell to accomplish this automation, and learned about key ECS and EC2 Auto Scaling commands.

Although you have a functional ECS Cluster at this point, you haven’t yet deployed any containers (ECS tasks) to it. Keep an eye out for part 2 of this blog post, where you’ll explore building your own custom container images, pushing those container images up to Amazon EC2 Container Registry (Amazon ECR), and running ECS tasks and services.

Writing and Archiving Custom Metrics using Amazon CloudWatch and AWS Tools for PowerShell

This is a guest post from Trevor Sullivan, a Seattle-based Solutions Architect at Amazon Web Services (AWS). Since 2004, Trevor has worked intimately with Microsoft technologies, including PowerShell since its release in 2006. In this article, Trevor takes you through the process of using the AWS Tools for PowerShell to write and export metrics data from Amazon CloudWatch.

Amazon’s CloudWatch service is an umbrella that covers a few major areas: logging, metrics, charting, dashboards, alarms, and events.

I wanted to take a few minutes to cover the CloudWatch Metrics area, specifically as it relates to interacting with metrics from PowerShell. We’ll start off with a discussion and demonstration of how to write metric data into CloudWatch, then move on to how to find existing metrics in CloudWatch, and finally how to retrieve metric data points from a specific metric.

Amazon CloudWatch stores metrics data for up to 15 months. However, you can export data from Amazon CloudWatch into a long-term retention tool of your choice, depending on your requirements for metric data retention, and required level of metric granularity. While historical, exported data may not be usable inside CloudWatch, after aging out, you can use other AWS data analytics tools, such as Amazon QuickSight and Amazon Athena to build reports against your historical data.


For this article, we assume that you have an AWS account. We also assume you understand PowerShell at a fundamental level, have installed PowerShell and the AWS Tools for PowerShell on your platform of choice, and have already set up your AWS credentials file and necessary IAM policies granting access to CloudWatch. We’ll discuss and demonstrate how to call various CloudWatch APIs from PowerShell, so be sure you’re prepared for this topic.

For more information, see the Getting Started guide for AWS Tools for PowerShell.

Write metric data into Amazon CloudWatch

Let’s start by talking about storing custom metrics in CloudWatch.

In the AWS Tools for PowerShell, there’s a command named Write-CWMetricData. This PowerShell command ultimately calls the PutMetricData API to write metrics to Amazon CloudWatch. It’s fairly easy to call this command, as there are only a handful of parameters. However, you should understand how CloudWatch works before attempting to use the command.

  • CloudWatch metrics are stored inside namespaces
  • Metric data points:
    • Must have a name.
    • May have zero or more dimensions.
    • May have a value, time stamp and unit of measure (eg. Bytes, BytesPerSecond, Count, etc.).
    • May specify a custom storage resolution (eg., 1 second, 5 seconds—the default is 60 seconds).
  • In the AWS Tools for PowerShell, you construct one or more MetricDatum .NET objects, before passing these into Write-CWMetricData.

With that conceptual information out of the way, let’s look at the simplest way to create a custom metric. Writing metric data points into CloudWatch is how you create a metric. There isn’t a separate operation to create a metric and then write data points into it.

### First, we create one or more MetricDatum objects
$Metric = [Amazon.CloudWatch.Model.MetricDatum]::new()
$Metric.MetricName = 'UserCount'
$Metric.Value = 98

### Second, we write the metric data points to a CloudWatch metrics namespace
Write-CWMetricData -MetricData $Metric -Namespace trevortest/tsulli.loc

If you have lots of metrics to track, and you’d prefer to avoid cluttering up your top-level metric namespaces, this is where metric dimensions can come in handy. For example, let’s say we want to track a “UserCount” metric for over 100 different Active Directory domains. We can store them all under a single namespace, but create a “DomainName” dimension, on each metric, whose value is the name of each Active Directory domain. The following screenshot shows an example of this in action.

Here’s a PowerShell code example that shows how to write a metric to CloudWatch, with a dimension. Although the samples we’ve looked at in this article show how to write a single metric, you should strive to reduce the number of disparate AWS API calls that you make from your application code. Try to consolidate the gathering and writing of multiple metric data points in the same PutMetricData API call, as an array of MetricDatum objects. Your application will perform better, with fewer HTTP connections being created and destroyed, and you’ll still be able to gather as many metrics as you want.

$Metric = [Amazon.CloudWatch.Model.MetricDatum]::new()
### Create a metric dimension, and set its name and value
$Dimension = [Amazon.CloudWatch.Model.Dimension]::new()
$Dimension.Name = 'DomainName'
$Dimension.Value = 'awstrevor.loc'

$Metric.MetricName = 'UserCount'
$Metric.Value = 76
### NOTE: Be sure you assign the Dimension object to the Dimensions property of the MetricDatum object
$Metric.Dimensions = $Dimension
Write-CWMetricData -MetricData $Metric -Namespace trevortest

Retrieve a list of metrics from Amazon CloudWatch

Now that we’ve written custom metrics to CloudWatch, let’s discuss how we search for metrics. Over time, you might find that you have thousands or tens of thousands of metrics in your AWS accounts, across various regions. As a result, it’s imperative that you know how to locate metrics relevant to your analysis project.

You can, of course, use the AWS Management Console to view metric namespaces, and explore the metrics and metric dimensions contained within each namespace. Although this approach will help you gain initial familiarity with the platform, you’ll most likely want to use automation to help you find relevant data within the platform. Automation is especially important when you introduce metrics across multiple AWS Regions and multiple AWS accounts, as they can be harder to find via a graphical interface.

In the AWS Tools for PowerShell, the Get-CWMetricList command maps to the AWS ListMetrics API. This returns a list of high-level information about the metrics stored in CloudWatch. If you have lots of metrics stored in your account, you might get back a very large list. Thankfully, PowerShell has some generic sorting and filtering commands that can help you find the metrics you’re seeking, with some useful filtering parameters on the Get-CWMetricList command itself.

Let’s explore a few examples of how to use this command.

Starting with the simplest example, we’ll retrieve a list of all the CloudWatch metrics from the current AWS account and region.


If the results of this command are little overwhelming, that’s okay. We can filter down the returned metrics to a specific metric namespace, using the -Namespace parameter.

Get-CWMetricList -Namespace AWS/Lambda

What if you don’t know which metric namespaces exist? PowerShell provides a useful command that enables you to filter for unique values.

(Get-CWMetricList).Namespace | Select-Object -Unique

If these results aren’t in alphabetical order, it might be hard to visually scan through them, so let’s sort them.

(Get-CWMetricList).Namespace | Select-Object -Unique | Sort-Object

Much better! Another option is to search for metrics based on a dimension key-value pair. It’s a bit more typing, but it’s a useful construct to search through thousands of metrics. You can even write a simple wrapper PowerShell function to make it easier to construct one of these DimensionFilter objects.

$Filter = [Amazon.CloudWatch.Model.DimensionFilter]::new()
$Filter.Name = 'DomainName'
$Filter.Value = 'tsulli.loc'
Get-CWMetricList -Dimension $Filter

If you know the name of a specific metric, you can query for a list of metrics that match that name. You might get back multiple results, if there are multiple metrics with the same name, but different dimensions exist in the same namespace. You can also have similarly named metrics across multiple namespaces, with or without dimensions.

Get-CWMetricList -MetricName UserCount

PowerShell’s built-in, generic Where-Object command is infinitely useful in finding metrics or namespaces, if you don’t know their exact, full name.

This example shows how to filter for any metric names that contain “User”.

Get-CWMetricList | Where-Object -FilterScript { $PSItem.Name -match 'User' }

Filtering metrics by namespace is just as easy. Let’s search for metrics that are stored inside any metric namespace that ends with “EBS”.

Get-CWMetricList | Where-Object -FilterScript { $PSItem.Namespace -match 'EBS$' }

That’s likely enough examples of how to find metrics in CloudWatch, using PowerShell! Let’s move on and talk about pulling actual metric data points from CloudWatch, using PowerShell.

Pull metric data from CloudWatch


Metric data is stored in CloudWatch for a finite period of time. Before metrics age out of CloudWatch, metric data points (metric “statistics”) move through a tiered system where they are aggregated and stored as less granular metric data points. For example, metrics gathered on a per-minute period are aggregated and stored as five-minute metrics, when they reach an age of fifteen (15) days. You can find detailed information about the aggregation process and retention period in the Amazon CloudWatch metrics documentation.

Data aggregation in CloudWatch, as of this writing, starts when metric data points reach an age of three (3) hours. You need to ensure that you’re exporting your metric data before your data is aged, if you want to keep the most detailed resolution of your metric data points. Services such as AWS Lambda or even PowerShell applications deployed onto Amazon EC2 Container Service (ECS) can help you achieve this export process in a scalable fashion.

The longest period that metrics are stored in CloudWatch is 15 months. If you want to store metrics data beyond a 15-month period, you must query the metrics data, before CloudWatch performs aggregation on your metrics, and store it in an alternate repository, such as Amazon DynamoDBAmazon S3, or Amazon RDS.

PowerShell Deep Dive

Now that we’ve covered some of the conceptual topics around retrieving and archiving CloudWatch metrics, let’s look at the actual PowerShell command to retrieve data points.

The AWS Tools for PowerShell include a command named Get-CWMetricStatistic, which maps to the GetMetricStatistics API in AWS. You can use this command to retrieve granular data points from your CloudWatch metrics.

There are quite a few parameters that you need to specify on this command, because you are querying a potentially massive dataset. You need to be very specific about the metric namespace, name, start time, end time, period, and statistic that you want to retrieve.

Let’s find the metric data points for the Active Directory UserCount metric, for the past 60 minutes, every minute. We assign the API response to a variable, so we can dig into it further. You most likely don’t have this metric, but I’ve been gathering this metric for awhile, so I’ve got roughly a week’s worth of data at a per-minute level. Of course, my metrics are subject to the built-in aggregation policies, so my per-minute data is only good for up to 15 days.

$Data = Get-CWMetricStatistic -Namespace ActiveDirectory/tsulli.loc -ExtendedStatistic p0.0 -MetricName UserCount -StartTime ([DateTime]::UtcNow.AddHours(-1)) -EndTime ([DateTime]::UtcNow) -Period 60

As you can see, the command is a little lengthy, but when you use PowerShell’s tab completion to help finish the command and parameter names, typing it out isn’t too bad.

Because we’re querying the most recent hour’s worth of data, we should have exactly 60 data points in our response. We can confirm this by examining PowerShell’s built-in Count property on the Datapoints property in our API response.


The data points aren’t in chronological order when returned, so let’s use PowerShell to sort them, and grab the most recent data point.

$Data.Datapoints | Sort-Object -Property Timestamp | Select-Object -Last 1
Average            : 0
ExtendedStatistics : {[p0.0, 19]}
Maximum            : 0
Minimum            : 0
SampleCount        : 0
Sum                : 0
Timestamp          : 10/14/17 4:27:00 PM
Unit               : Count

Now we can take this data and start exporting it into our preferred external data store, for long-term retention! I’ll leave it to you to explore this API further.

Due to the detailed options available in the GetMetricStatistics API, I would strongly encourage you to read through the documentation, and more importantly, run your own experiments with the API. You need to use this API extensively, if you want to export data points from CloudWatch metrics to an alternative data source, as described earlier.


In this article, we’ve explored the use of the AWS Tools for PowerShell to assist you with writing metrics to Amazon CloudWatch, searching or querying for metrics, and retrieving metric data points. I would encourage you to think about how Amazon CloudWatch can integrate with a variety of other services, to help you achieve your objectives.

Don’t forget that after you’ve stored metrics data in CloudWatch, you can then build dashboards around that data, connect CloudWatch alarms to notify you about infrastructure and application issues, and even perform automated remediation tasks. The sky is the limit, so put on your builder’s hat and start creating!

Please feel free to follow me on Twitter, and keep an eye on my YouTube channel.