AWS Security Blog

Sharing AWS CloudTrail Log Files Between Accounts

If you use AWS CloudTrail to log API calls in your account, you can share your log files with other AWS accounts, whether you own those accounts or not. In this post, Greg Pettibone, a technical writer on the CloudTrail team, walks through some cross-account scenarios to show you how.

AWS CloudTrail captures information about all the API calls made to supported services in your AWS account. For example, if user Dave starts or stops an Amazon EC2 instance in your account, or if user Alice creates an IAM user or changes the user’s password, these actions are recorded by CloudTrail. A CloudTrail log entry tells you what request was made (the action and the requested resource), the source IP address for the request, who made the request, when it was made, and so on.

In the simplest case, CloudTrail writes information to an Amazon S3 bucket that belongs to the AWS account where CloudTrail is turned on. But there are a couple of scenarios where it’s useful to share the S3 bucket that contains CloudTrail log files with other accounts:

  • You want to aggregate log information from multiple AWS accounts that you own into a single S3 bucket.
  • You want to grant access to the S3 bucket to an account owned by someone else. For example, you might be working with a vendor who will read your CloudTrail log files and perform analysis on the data.

In this post, I’ll walk you through these scenarios.

[Update (Sept 24, 2014): Made corrections to two steps in the following procedures.]

Prerequisites: One AWS account, one Amazon S3 bucket, and CloudTrail

Before you can share log files, CloudTrail must be able to deliver those files to an Amazon S3 bucket that you own. You begin by turning on CloudTrail in your AWS account and specifying a bucket to use. We recommend that you choose the default option in CloudTrail, which is to create a new Amazon S3 bucket, so that the correct bucket access policies are automatically created for you.

For example, imagine that Example Corp has an AWS account named Production with the ID number 123456789012, and that Alice is the administrator for that account. Alice turns on CloudTrail in the Production account and specifies that CloudTrail should write log files to a new bucket named example-ct-logs. She performs the following actions:

  1. She logs into the AWS Management Console using her Production account password.
  2. She opens the CloudTrail console and clicks Get Started.
  3. She accepts the default value of Yes for Create a new S3 Bucket?, enters the name example-ct-logs, and then clicks Advanced. Although Alice could have CloudTrail write logs to an existing bucket, we’ll assume she wants to create a new bucket for the log files.
  4. She does not enter a log file prefix. A custom prefix would let Alice add a layer to the path so that she could further segregate log file locations, but we’ll assume that Alice considers the default location good enough.
  5. She chooses Yes for Include global services? This makes sure that CloudTrail log files include information about activity with services like AWS IAM and AWS STS.
  6. She accepts the default of No for SNS notifications for every log file delivery? We’ll assume that Alice doesn’t want to be notified each time a new log file is written to the Amazon S3 bucket.
  7. She clicks Subscribe.

Here’s the scenario that Alice has set up:

Diagram showing the scenario Alice set up

CloudTrail will now write log files into the bucket using a path that looks like this:

example-ct-logs/AWSLogs/account-id/CloudTrail/region/year/month/day

Notice that the bucket name, example-ct-logs, is the fixed literal string that Alice specified when she turned on CloudTrail. The two other literal strings (AWSLogs and CloudTrail) are assigned by CloudTrail and appear in every Amazon S3 path that CloudTrail creates. For example, assume that Alice turned on the Production account (123456789012) in the Northern Virginia region (us-east-1) on July 31, 2014. Assuming that there’s activity in the account that can be recorded, within 15 minutes Alice will see log files begin to appear in the bucket. They’ll be created as objects that in the following path:

example-ct-logs/AWSLogs/123456789012/CloudTrail/us-east-1/2014/07/31

The following day, log activity, if any, will be delivered to files in a path that looks like this:

example-ct-logs/AWSLogs/123456789012/CloudTrail/us-east-1/2014/08/01

The paths will change in this manner every day that there is log activity unless Alice turns CloudTrail off.

Sharing scenario 1: Aggregate log files for the Dev account into the Production account’s bucket

Now suppose that Example Corp has another AWS account named Dev whose account ID is 999988887777. Alice is also the administrator for this account and wants to turn on CloudTrail for it. She could do the same thing she did for the Production account and create a new Amazon S3 bucket for the Dev account, but she decides that it’s more convenient to have log files from both accounts written to the bucket that she created for the Production account. That is, Alice intends to aggregate log files from both accounts into the example-ct-logs bucket, which is owned by the Production account. This is convenient because she needs to go to only one bucket to look at log files from either the Production or Dev account. Alice’s aggregation scenario looks like this:

Diagram showing Alice's aggregation scenario

To keep things straight, CloudTrail will deliver log files into different folders in the same Amazon S3 bucket. We’ve already established, for example, that on July 31, 2014, CloudTrail will deliver log files for the Production account to the following folder:

example-ct-logs/AWSLogs/123456789012/CloudTrail/us-east-1/2014/07/31

Once everything is set up properly, CloudTrail will also deliver log files for the Dev account to the following folder, which reflects the account ID for the Dev account.

example-ct-logs/AWSLogs/999988887777/CloudTrail/us-east-1/2014/07/31

Editing the bucket access policy

However, before Alice can enable log file aggregation across accounts, she must edit the bucket policy on the example-ct-logs bucket. When she originally turned on CloudTrail in the Production account, CloudTrail added a policy to the Amazon S3 bucket that enabled the CloudTrail to write to the bucket owned by the Production account (123456789012)[corrected 24-Sep-14]. The only change that Alice needs to make in the bucket policy is to allow AWS accounts to write into a new folder, this one for the Dev account (therefore, 999988887777). Alice does the following:

  1. She signs into the Amazon S3 console by using credentials for the Production account.
  2. She selects the example-ct-logs bucket and clicks Properties and then Permissions.
  3. She clicks Edit Bucket Policy.
    Screenshot showing "Edit bucket policy" option
  4. She edits the policy, and in the Resource field, she adds a line that specifies a folder for the Dev account, like the highlighted line in this example:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AWSCloudTrailAclCheck20131101",
      "Effect": "Allow",
      "Principal": {
                        "Service": "cloudtrail.amazonaws.com"
                    },
      "Action": "s3:GetBucketAcl",
      "Resource": "arn:aws:s3:::my-cloudtrail-logs"
    },
    {
      "Sid": "AWSCloudTrailWrite20131101",
      "Effect": "Allow",
      "Principal": {
                        "Service": "cloudtrail.amazonaws.com"
                    },
      "Action": "s3:PutObject",
      "Resource": [
        "arn:aws:s3:::my-cloudtrail-logs/AWSLogs/123456789012/*",
        "arn:aws:s3:::my-cloudtrail-logs/AWSLogs/999988887777/*"
      ],
      "Condition": {
        "StringEquals":
          {"s3:x-amz-acl": "bucket-owner-full-control"}
      }
    }
  ]
}

This is the only change she needs to make in the policy.

Turning on CloudTrail in the Dev account

Now Alice can turn on CloudTrail in the Dev account and point the log files to the shared bucket. She does the following:

  1. She signs into the AWS Management Console by using credentials for the Dev account.
  2. In the navigation bar, she selects the region in the Dev account where she wants CloudTrail turned on.
  3. She clicks Get Started, and when she’s prompted to create a new S3 bucket, she selects No. This is where the process diverges from when she first turned CloudTrail on in the Production account.
  4. In the text box, she enters example-ct-logs as the bucket to which log files will be delivered.
  5. As she did with the Production account, she clicks Advanced, and for Include Global Services she selects Yes. This causes CloudTrail to record events for global services such as IAM or AWS STS when they occur in the Dev account.[corrected 24-Sep-2014]
  6. For SNS notifications, she selects No, as she did for the Production account.
    Screenshot of choosing not to receive SNS notifications
  7. She clicks Subscribe.

From now on, whenever an API call is made in the Dev account, the information is logged in the example-ct-logs bucket. Since CloudTrail uses a different folder in the bucket for each account and each date, once log files start appearing in example-ct-logs, Alice sees paths similar to these in her Production S3 console:

example-ct-logs/AWSLogs/123456789012/CloudTrail/us-east-1/2014/07/31
example-ct-logs/AWSLogs/123456789012/CloudTrail/us-east-1/2014/08/01
example-ct-logs/AWSLogs/123456789012/CloudTrail/us-east-1/2014/08/02
...
example-ct-logs/AWSLogs/999988887777/CloudTrail/us-east-1/2014/07/31
example-ct-logs/AWSLogs/999988887777/CloudTrail/us-east-1/2014/08/01
example-ct-logs/AWSLogs/999988887777/CloudTrail/us-east-1/2014/08/02
...

Sharing scenario 2: Share read-only access to CloudTrail log files with a third party

Once log files start accumulating in the example-ct-logs bucket, Example Corp decides that they want to analyze the log information. They hire AnyCompany, a vendor that provides a wide variety of reports and tools, to help Example Corp see what sorts of information the log files are capturing.

Since there’s a lot of AWS activity at Example Corp across the accounts that have CloudTrail turned on, the log files accumulate pretty quickly. Therefore, it makes the most sense to give AnyCompany direct, read-only access to the example-ct-logs bucket.

The best (and recommended) way for Example Corp to share AWS CloudTrail log files with a vendor is to use cross-account access via an IAM role. This lets AnyCompany get temporary security credentials that they can use to access the bucket that contains the log files.

The scenario is illustrated by the following diagram:

Diagram illustrating Scenario 2

This approach assumes that AnyCompany has its own AWS account. Since AnyCompany accesses AWS resources on behalf of the resource owner, AnyCompany should also have an external ID for ExampleCorp. This is a value that an AWS vendor gives to its customers that uniquely identifies the customer in the vendor’s system. I’ll explain shortly how the external ID is used.

Alice must now configure access for AnyCompany to the S3 bucket example-ct-logs. She contacts the vendor to get two pieces of information:

  • The vendor’s 12-digit AWS account ID. Let’s assume that AnyCompany’s account ID is 998877665544.
  • The external ID that AnyCompany has assigned to Example Corp. This might be a customer number or some other code. Alice doesn’t really know, and it’s not important, other than that AnyCompany has to be sure that the external ID they provide to Alice is unique among AnyCompany customers. Note that Alice must have an external ID from AnyCompany in order to use the IAM console to set up cross-account access.

To grant AnyCompany access to the log files that CloudTrail delivers to the example-ct-logs bucket, Alice must create an IAM role in the Production account. This role requires two policies. The first is the trust policy, where Alice specifies that Example Corp trusts AnyCompany to assume the role and to get temporary security credentials to access the AWS Production account. The second is an access policy that specifies what actions the role allows AnyCompany to perform. In this case, of course, the access policy will permit AnyCompany read-only access to the example-ct-logs bucket. Optionally, Alice can scope the permissions down so that AnyCompany has access to only specific folders within the bucket.

Once Alice has the account ID and the external ID from AnyCompany, she does the following:

  1. Alice signs into the IAM console for the Production account.
  2. In the navigation pane, she clicks Roles and then Create New Role.
  3. She enters the name CloudTrailAccessForAnyCompany for the new role and clicks Next Step.
  4. She selects Role for Cross-Account Access.
  5. She selects Allow IAM users from a 3rd party AWS account to access this account.
    Screenshot of selecting "Allow IAM users from a 3rd party AWS account to access this account"
  6. Alice enters the 12-digit account ID for AnyCompany and the the external ID that AnyCompany has assigned to Example Corp, and then clicks Next Step.
    Screenshot of typing the account ID and external ID
  7. In the Set Permissions step, Alice clicks Custom Policy and then clicks Select.
  8. She enters the policy name ExampleCorp_CloudTrail_log_role and then copies the following policy into the Policy Document box. 
    {                                             
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": ["s3:Get*", "s3:List*"],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::my-cloudtrail-logs"],
      "Condition":{"StringLike":{"s3:prefix":["AWSLogs/*"]}}
    },
    {
       "Action":["s3:Get*", "s3:List*"],
       "Effect":"Allow",
       "Resource": ["arn:aws:s3:::my-cloudtrail-logs/AWSLogs/*"]
    }
  ]
}

    In order to be able to read the log files from example-ct-logs bucket, AnyCompany needs to perform two actions. The first action is GET Bucket (List Objects), which lets AnyCompany get all the keys in a bucket or in a folder (prefix) in the bucket. The second action that AnyCompany needs permission for is GET Object, which lets the vendor read individual objects from the bucket or from a folder in the bucket.

    This access policy gives the role permission to perform S3 Get* and List* actions. The permissions are scoped so that access is granted only to objects in the example-ct-logs bucket that have the AWSLogs/ prefix. Alice could skip the condition for the AWSLogs/ prefix and give the vendor access to the entire bucket, but she’s following the recommended practice of granting least privilege. In this case, AnyCompany needs access only to objects in example-ct-logs that have the AWSLogs/ prefix.

  9. Alice clicks Next Step and makes a note of the role ARN, which will look like this: 
    arn:aws:iam::123456789012:role/CloudTrailAccessForAnyCompany
  10. Alice clicks Create Role.

That’s it for creating the role. Because Alice used the IAM console to create the role, the trust policy was created for her and looks like this:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"AWS": "arn:aws:iam::998877665544:root"},
    "Action": "sts:AssumeRole",
    "Condition": {"StringEquals": {"sts:ExternalId": "8900-544"}}
  }]
}

In this trust policy, the Principal element is set to the vendor’s account ID. This means that Example Corp is trusting AnyCompany to assume the role that Alice just created. However, you can see that there’s a condition on the policy, and AnyCompany can only assume this rule if they pass the external ID they’ve assigned to Example Corp as part of the request to assume this role.

Once Alice has finished creating the role, she contacts AnyCompany and gives them the following information:

  • The ARN of the role that they can assume. In the example, that’s arn:aws:iam::123456789012:role/ExampleCorp-CloudTrail-log-role.
  • The name of the bucket (here, example-ct-logs) where the vendor can find the log files.
  • The prefix that CloudTrail uses in the bucket (AWSLogs/).

Someone who has administrator permissions at AnyCompany in turn has to delegate permission to one or more individual users in their own account to assume the role. A user at AnyCompany who’s been granted the right permissions (or a process that has access to user credentials) can then programmatically assume the role and read log files from the bucket.

Conclusion

Both scenarios I discussed involve cross-account access for CloudTrail log files. The first scenario shows you how to use a bucket policy to let CloudTrail write to a central log file from multiple AWS accounts that you own. The second scenario shows you how to use a role to let a third party access your log files. Both scenarios are common scenarios for working with CloudTrail.

If you’re not already using CloudTrail, we recommend that you turn it on in all of your AWS accounts—in fact, using CloudTrail is one of our recommended best practices. And of course, once you start keeping logs, you can review them periodically to analyze usage in your account or accounts. To help you with this task, there are a number of AWS Partners that specialize in logging and analysis to provide solutions that leverage CloudTrail output. For more information, visit the CloudTrail detail page at http://aws.amazon.com/cloudtrail.

More Information

If you’d like more information about CloudTrail or about the scenarios we examined, see the following documentation:

If you have questions, please visit the CloudTrail forum.

-Greg