AWS News Blog

AWS CloudTrail Update – SSE-KMS Encryption & Log File Integrity Verification

My colleague Sivakanth Mundru sent the guest post below to introduce a pair of features for AWS CloudTrail.

— Jeff;

As you know, AWS CloudTrail records API calls made on your account and delivers log files containing API activity to an S3 bucket you specify. Today, we are announcing two new features for CloudTrail:

  • Support for Encryption using SSE-KMS – You can add an additional layer of security for the CloudTrail log files stored in your S3 bucket by encrypting them with your AWS Key Management Service (AWS KMS) key. CloudTrail will encrypt the log files using the KMS key you specify.
  • Log File Integrity Validation – You can validate the integrity of the CloudTrail log files stored in your S3 bucket and detect whether they were deleted or modified after CloudTrail delivered them to your S3 bucket. You can use the log file integrity (LFI) validation as a part of your security and auditing discipline.

These features are available today in the US East (N. Virginia), US West (Oregon), US West (N. California), Europe (Ireland), Europe (Frankfurt), Asia Pacific (Tokyo), Asia Pacific (Sydney), Asia Pacific (Singapore), and South America (São Paulo) regions.

Support for Encryption Using KMS
CloudTrail generates log files and sends them to a S3 bucket. By default, the files are encrypted using S3’s Server Side Encryption (SSE), and then transparently decrypted when you read them. With today’s launch you can now provide a KMS key to CloudTrail and it will be used to encrypt your log files. As is the case with SSE, decryption is transparent and automatic if you have permission to read the object. Therefore, applications that read and process log files do not require any changes. You simply need to give S3 permission to decrypt the files. Here’s how it all fits together:

Here’s how you can set this up yourself:

  1. Create a KMS key or use an existing KMS key in the same region as the S3 bucket where you receive your CloudTrail log files and apply the KMS-CloudTrail policy.
  2. Apply decrypt permissions to the principal (IAM users, roles, groups, and so forth) that will be accessing the CloudTrail log files.
  3. Update an existing trail with the KMS key from step 1 (you can enable encryption at the time you create a trail if you use the CLI).

Log File Integrity Validation
If you are doing a security audit or investigation, you may want to validate the integrity of the CloudTrail log files stored in your S3 bucket and detect whether they been deleted or modified since CloudTrail delivered the log file to your S3 bucket (the expectation is that they will be unchanged). The new CloudTrail log file integrity validation feature enables you do that.

In order to validate the integrity of log files, you need to enable log file validation for your trail.  You can do this by setting Enable log file validation to Yes in the advanced section of your trail configuration:

Once you enable log file integrity validation, CloudTrail will start delivering digest files, on an hourly basis, to the same S3 bucket where you receive your CloudTrail log files, but with a different prefix:

  • CloudTrail log files are delivered to /optional_prefix/AWSLogs/AccountID/CloudTrail/*.
  • CloudTrail digest files are delivered to /optional_prefix/AWSLogs/AccountID/CloudTrail-Digest/*.

This layout allows applications that integrate with CloudTrail to process the log files without making any changes. You can also apply different and granular access control permission to the log files and digest files.

The digest files contain information about the log files that were delivered to your S3 bucket, hash values for those log files, digital signatures for the previous digest file and the digital signature for the current digest file in the S3 metadata section. For more information about digest files, digital signatures and hash values, read about the CloudTrail Digest File Structure.

To validate the CloudTrail log files, use the AWS Command Line Interface (AWS CLI) and simply run the following command to validate the log files:

$ aws cloudtrail validate-logs \
  --trail-arn arn:aws:cloudtrail:us-west-2:111111111111:trail/Trailname \
  --start-time 2015-09-24T00:00:00Z --region=us-west-2

If the log files have not been modified or deleted you will see output that looks like this:

Validating log files for trail arn:aws:cloudtrail:us-west-2:111111111111:trail/Trailname between \
  2015-09-24T00:00:00Z and 2015-09-25T18:56:41Z
Results requested for 2015-09-24T00:00:00Z to 2015-09-25T18:56:41Z
Results found for 2015-09-24T00:30:26Z to 2015-09-25T18:56:41Z:
43/43 digest files valid
31/31 log files valid

If one or more log files have been deleted you will see output that looks like this:

Log file s3://mybucket-CTlogs/AWSLogs/111111111111/CloudTrail/us-west-2/2015/09/22/111111111111_CloudTrail_us-west-2_20150922T1720Z_Jy4SwZotr3eTI2FM.json.gz \
       INVALID: not found
Results requested for 2015-09-22T00:00:00Z to 2015-09-25T18:42:03Z
Results found for 2015-09-22T00:30:26Z to 2015-09-25T18:42:03Z:
43/43 digest files valid
30/31 log files valid, 1/31 log files INVALID

If one or more log files have been modified you will see output that looks like this:

Log file s3://mybucket-CTlogs/AWSLogs/111111111111/CloudTrail/us-west-2/2015/09/25/111111111111_CloudTrail_us-west-2_20150925T1845Z_lU58MiCsXyI1U3R1.json.gz \
       INVALID: hash value doesn't match
Results requested for 2015-09-24T00:00:00Z to 2015-09-25T21:44:50Z
Results found for 2015-09-24T00:30:26Z to 2015-09-25T21:44:50Z:
45/45 digest files valid
35/36 log files valid, 1/36 log files INVALID

You can run the validate-logs command in verbose mode to perform a deeper analysis.

To learn more about this feature, read about Validating CloudTrail Log File Integrity.

If you have any questions or feedback on these new features, you can post them in the CloudTrail forums.

— Sivakanth Mundru, Senior Product Manager, AWS CloudTrail