Response Logging in AWS Tools for Windows PowerShell

As described in an earlier post, the AWS SDK for .NET has support for logging service responses, error responses, and metrics for AWS API calls. For the SDK, this is enabled through the App.config or Web.config file.

The AWS Tools for Windows PowerShell supports a shell variable, named $AWSHistory, that records what cmdlets have been run and the corresponding service response (and optionally request) data. However, until recently developers wanting to use configurable and more detailed diagnostic logging from within the underlying SDK were only able to effect this by editing the configuration file for PowerShell itself (powershell.exe.config)—which affects logging for all PowerShell scripts.

We recently added a cmdlet that makes it possible to configure logging with System.Diagnostics within a script. This cmdlet affects only the currently running script. It will either create simple TextWriterTraceListener instances, or allow you to add custom listeners for the trace sources associated with AWS requests.

First, let’s add a simple text listener:

Add-AWSLoggingListener MyAWSLogs c:logsaws.txt

This listener creates a TextWriterTraceListener that logs error responses from AWS requests to the file c:logsaws.txt. The listener is attached to the source Amazon, which matches all service requests.

If we want to send Amazon S3 errors to a separate log, we could add a second listener:

Add-AWSLoggingListener MyS3Logs c:logss3.txt -Source Amazon.S3

Trace data will go only to the most-specific trace source configured for a listener. In this example, the S3 logs go to s3.txt and all other service logs go to aws.txt.

By default, listeners added in this way will log only error responses. Enabling logging of all responses and/or metrics can be done with a couple of other cmdlets:

Set-AWSResponseLogging Always
Enable-AWSMetricsLogging

These cmdlets affect all listeners added with Add-AWSLoggingListener. Similarly, we can turn those logging levels back down or off:

Set-AWSResponseLogging OnError
Set-AWSResponseLogging Never
Disable-AWSMetricsLogging

Also, we can remove specific listeners from a trace source by name:

Remove-AWSLoggingListener Amazon MyAWSLog

Now, only the S3 logger is active. One way you could use these cmdlets is to enable logging only around a particular section of script.

The Add-AWSLoggingListener cmdlet can also add instances of trace listeners created by other means, such as custom listeners. These statements do the same thing:

Add-AWSLoggingListener -Name MyAWSLog -LogFilePath c:logsaws.txt

$listener = New-Object System.Diagnostics.TextWriterTraceListener c:logsaws.txt
$listener.Name = "MyAWSLog"
Add-AWSLoggingListener -TraceListener $listener

Exposing this facility through the PowerShell cmdlets required adding the ability to programmatically add or remove listeners via the existing AWSConfigs class in the AWS SDK for .NET, in addition to the logging-related configuration items already on that class.

AWSConfigs.AddTraceListener("Amazon.DynamoDB", 
  new TextWriterTraceListener("c:\logs\dynamo.txt", "myDynamoLog"));

Now PowerShell developers have the same access to performance and diagnostic information about AWS API calls as other AWS SDK for .NET users. For more information, refer to the Shell Configuration section of the AWS Tools for Windows PowerShell Cmdlet Reference.