Copy existing AWS CloudTrail trails events to a AWS CloudTrail Lake event data store
AWS announced the general availability of AWS CloudTrail Lake on 5th Jan 2022, a managed audit and security lake that lets you aggregate, immutably store, and query activity logs for auditing, security investigation, and operational troubleshooting. Since launch, customers have adopted this feature, and it’s an integral part of customer operational and security operational processes.
Today, we’re excited to announce a new import feature which will allow customers to copy existing trail events to a CloudTrail Lake event data store which were recorded prior to event data store creation. This new feature allows customers to import events from an Amazon Simple Storage Service (Amazon S3) buckets corresponding to a trail during auditing or troubleshooting. This also improves the efficiency during auditing, as well as security or operational incident troubleshooting. You can utilize SQL query language from a single source to find the required information. You no longer need to maintain multiple trails or data processing platforms. This new feature supports copying logs from an S3 bucket that stores logs from across multiple accounts (from an organization trail) and multiple AWS Regions. Furthermore, you can import logs from individual accounts and single-region trails. This feature will also let you specify an import date range, so that you only import the subset of logs that are needed for long-term storage and analysis in Lake.
In this post, I’ll walk you through this import feature that allows you to copy existing trail events to a CloudTrail Lake event data store. If you’re using CloudTrail Lake for the first time, then check this post.
Importing CloudTrail events from Trails page in CloudTrail console
- Open the AWS Console and sign in with an account with administrative permissions to manage CloudTrail.
- Navigate to CloudTrail console. Choose Trails in the left navigation pane of the CloudTrail console.
- Choose the trail that you want to copy and select Copy events to Lake.
- (Optional) Choose the time range for importing events.
- Select Lake event data store to copy trail events.
- Select the AWS Identity and Access Management (IAM) role to copy events. You can either create a new role or select an existing role with adequate permission. You can read more about permissions required to copy trail events.
- Select copy events.
- Check the status of the copying job by viewing the event data store.
Now we’ve seen how to copy events from an existing trail into an event data store. Alternatively, you can copy events from Amazon S3, from the CloudTrail lake event data store page.
Importing CloudTrail events from the event data store details page in the CloudTrail console
- Open the Console and sign in with an account with administrative permissions to manage CloudTrail.
- Navigate to the CloudTrail console. Choose Lake in the left navigation pane of the CloudTrail console, and then choose Event data stores.
- On the Event data stores page, choose Copy trail events.
- Select the trail that you want to copy in the Events source drop-down list.
- (Optional) Choose the time range for copying events.
- Choose the event data store from the drop-down list to copy trail events.
- For Permissions, create a new role to copy trail events, or select an existing role from the drop-down list.
- Choose Copy events to start importing.
- On the Event copy status page, review the status for copying.
- Review the details of the copy status.
- Copy status details will show the following details of the trail event copy:
- Event S3 location
- S3 prefix copied
- Copy status
- Copy error details like error message, error location and error type
After the event copy status is complete, run an SQL query in the event data store to analyze events.
To get started, see Working with CloudTrail Lake Import feature in the CloudTrail User Guide.
In the post, we’ve announced the new CloudTrail Lake import feature. We’ve shown you how to copy events from CloudTrail trails directly, or from event data stores. To get started, see Working with CloudTrail Lake in the CloudTrail User Guide. Also, refer CloudTrail Lake getting started blog which demonstrate setting CloudTrail Lake and performing few example queries. We’re excited to make this new feature available for you, and we can’t wait to see what great things you build with it.
About the author: