AWS Developer Blog

Configuring Advanced Logging on AWS Elastic Beanstalk

by Jim Flanagan | on | in .NET | Permalink | Comments |  Share

Sometimes developers want more flexibility in logging for their IIS environments. For example, in the IIS log files on instances in a load-balanced AWS Elastic Beanstalk environment, the client IP for requests always appears to be the load balancer. Elastic Load Balancing adds an X-Forwarded-For header to each request that contains the actual client IP address, but there’s no way to log that with IIS’s default logging.

Microsoft has created Advanced Logging to provide more flexibility in logging. You can add Advanced Logging to your Elastic Beanstalk instances by making the MSI available (for example, in an Amazon S3 bucket), then scripting the configuration with Windows PowerShell.

By default, though, Advanced Logging puts its log files in a different location than the default IIS log files, so if you want to see them in Snapshot Logs, or have them published to S3, you need to tell Elastic Beanstalk about it.

We’ll build up an .ebextensions config file that addresses each of these points, then show the completed configuration at the end.

Download and Install Advanced Logging

First, we need to make the Advanced Logging installer available at a well-known location that will persist over the lifetime of our environment, because instances that get autoscaled into the environment will need to download it as well as any instances created when the environment is first brought up. This can be any URL-addressable location. For this example, we will upload the installer to an S3 bucket we control.

After uploading the AdvancedLogging64.msi to an S3 bucket and making it publicly readable, add the following to the config file (for example: .ebextensionsadvancedlogging.config).

files:
  "c:/software/AdvancedLogging64.msi": 
  source: https://my-bucket.s3.amazonaws.com/AdvancedLogging64.msi
commands:
  00-install-advanced-logging:
    command: msiexec /i AdvancedLogging64.msi
    test: cmd /c "if exist c:\software\configured (exit 1) else (exit 0)"
    cwd: c:/software/
    waitAfterCompletion: 0
  02-set-configured:
    command: date /t > c:/software/configured
    waitAfterCompletion: 0

The files: key gets the MSI onto the instance, and the commands: key runs msiexec and then creates a file to signal that the install has been done. The test: subkey makes the command contingent on the non-existence of the signal file, so that the install happens only on the initial deployment, and not on every redeployment.

Configuring Advanced Logging

Advanced Logging is usually configured through the IIS Manager UI, but we can use PowerShell to accomplish this. The steps are:

  • disable IIS logging
  • add the X-Forwarded-For header to the list of possible fields
  • add X-Forwarded-For to the selected fields
  • enable Advanced Logging
  • iisreset

The PowerShell script to do this uses the WebAdministration module:

import-module WebAdministration

Set-WebConfigurationProperty `
  -Filter system.webServer/httpLogging `
  -PSPath machine/webroot/apphost `
  -Name dontlog `
  -Value true

Add-WebConfiguration "system.webServer/advancedLogging/server/fields" `
  -value @{id="X-Forwarded-For";sourceName="X-Forwarded-For";sourceType="RequestHeader";logHeaderName="X-Forwarded-For";category="Default";loggingDataType="TypeLPCSTR"}

$logDefinitions = Get-WebConfiguration "system.webServer/advancedLogging/server/logDefinitions"
foreach ($item in $logDefinitions.Collection) {
    Add-WebConfiguration `
      "system.webServer/advancedLogging/server/logDefinition/logDefinition[@baseFileName='$($item.baseFileName)']/selectedFields" `
      -value @{elementTagName="logField";id="X-Forwarded-For";logHeaderName="";required="false";defaultValue=""}
}

Set-WebConfigurationProperty `
  -Filter system.webServer/advancedLogging/server `
  -PSPath machine/webroot/apphost `
  -Name enabled `
  -Value true

iisreset

We can either put this script in a file and download it like the MSI, or just inline it in the config file, like this (without the added line breaks for readability):

files:
  "c:/software/configureLogging.ps1":
    content: |
      import-module WebAdministration
      Set-WebConfigurationProperty -Filter system.webServer/httpLogging -PSPath machine/webroot/apphost -Name dontlog -Value true
      Add-WebConfiguration "system.webServer/advancedLogging/server/fields" -value @{id="X-Forwarded-For";sourceName="X-Forwarded-For";sourceType="RequestHeader";logHeaderName="X-Forwarded-For";category="Default";loggingDataType="TypeLPCSTR"}
      $logDefinitions = Get-WebConfiguration "system.webServer/advancedLogging/server/logDefinitions"
      foreach ($item in $logDefinitions.Collection) {
        Add-WebConfiguration "system.webServer/advancedLogging/server/logDefinitions/logDefinition[@baseFileName='$($item.baseFileName)']/selectedFields" -value @{elementTagName="logField";id="X-Forwarded-For";logHeaderName="";required="false";defaultValue=""}
      }
      Set-WebConfigurationProperty -Filter system.webServer/advancedLogging/server -PSPath machine/webroot/apphost -Name enabled -Value true
      iisreset
commands:
  01-add-forwarded-header:
    command: Powershell.exe -ExecutionPolicy Bypass -File c:\software\configureLogging.ps1
    test: cmd /c "if exist c:\software\configured (exit 1) else (exit 0)"
    waitAfterCompletion: 0

This snippet creates the script file and executes it.

Configure Elastic Beanstalk Logging

As we mentioned before, the default location for Advanced Logging log files is different from where the IIS logs usually go. In order to get the Advanced Logging log files to show up for Snapshot Logs and log publication, we need to add some configuration files that tell the Snapshot Logs and log publication features where to look for log files. In this case, these files say that all files in C:inetpublogsAdvancedLogs are eligible for snapshotting or log publication.

files:
  "c:/Program Files/Amazon/ElasticBeanstalk/config/publogs.d/adv-logging.conf":
    content: |
      C:inetpublogsAdvancedLogs
  "c:/Program Files/Amazon/ElasticBeanstalk/config/taillogs.d/adv-logging.conf":
    content: |
      C:inetpublogsAdvancedLogs

Combining all of the above snippets into a single configuration file looks like this:

files:
  "c:/software/AdvancedLogging64.msi": 
    source: https://my-bucket.s3.amazonaws.com/AdvancedLogging64.msi
  "c:/Program Files/Amazon/ElasticBeanstalk/config/publogs.d/adv-logging.conf":
    content: |
      C:inetpublogsAdvancedLogs
  "c:/Program Files/Amazon/ElasticBeanstalk/config/taillogs.d/adv-logging.conf":
    content: |
      C:inetpublogsAdvancedLogs
  "c:/software/configureLogging.ps1":
    content: |
      import-module WebAdministration
      Set-WebConfigurationProperty -Filter system.webServer/httpLogging -PSPath machine/webroot/apphost -Name dontlog -Value true
      Add-WebConfiguration "system.webServer/advancedLogging/server/fields" -value @{id="X-Forwarded-For";sourceName="X-Forwarded-For";sourceType="RequestHeader";logHeaderName="X-Forwarded-For";category="Default";loggingDataType="TypeLPCSTR"}
      $logDefinitions = Get-WebConfiguration "system.webServer/advancedLogging/server/logDefinitions"
      foreach ($item in $logDefinitions.Collection) {
        Add-WebConfiguration "system.webServer/advancedLogging/server/logDefinitions/logDefinition[@baseFileName='$($item.baseFileName)']/selectedFields" -value @{elementTagName="logField";id="X-Forwarded-For";logHeaderName="";required="false";defaultValue=""}
      }
      Set-WebConfigurationProperty -Filter system.webServer/advancedLogging/server -PSPath machine/webroot/apphost -Name enabled -Value true
      iisreset
commands:
  00-install-advanced-logging:
    command: msiexec /i AdvancedLogging64.msi
    test: cmd /c "if exist c:\software\configured (exit 1) else (exit 0)"
    cwd: c:/software/
    waitAfterCompletion: 0
  01-add-forwarded-header:
    command: Powershell.exe -ExecutionPolicy Bypass -File c:\software\configureLogging.ps1
    test: cmd /c "if exist c:\software\configured (exit 1) else (exit 0)"
    waitAfterCompletion: 0
  02-set-configured:
    command: date /t > c:/software/configured
    waitAfterCompletion: 0

For more information about how to customize Elastic Beanstalk environments, see the AWS Elastic Beanstalk Developer Guide.