How do I stream log data from CloudWatch Logs to a cross-Region and cross-account Kinesis data stream?
Last updated: 2020-11-09
I need to send log data from Amazon CloudWatch Logs to another AWS account’s Amazon Kinesis data stream in a different AWS Region. How can I do this?
If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.
Important: To deliver CloudWatch log events to Kinesis data streams in different AWS accounts and Regions, set up cross-account log data sharing with subscriptions while specifying the AWS Region as follows.
In this example, CloudWatch Logs in the us-east-1 Region are delivered to another AWS user's Kinesis data stream in us-west-2.
1. Create a destination data stream in Kinesis in the data recipient account with an AWS Identity and Access Management (IAM) role and trust policy.
Specify the --region when you use the create-stream command to create the data stream. For example, this command creates the data stream YourStreamName in us-west-2:
>aws kinesis create-stream --stream-name "YourStreamName" --shard-count 1 --region us-west-2
Specify the --region when you use the describe-stream command to check the StreamDescription.StreamStatus property. For example, this command checks the stream YourStreamName in us-west-2:
>aws kinesis describe-stream --stream-name "YourStreamName" --region us-west-2
When you use the put-destination command to create the CloudWatch Logs destination, set the --region for the --role-arn to the same AWS Region as the source CloudWatch logs. For example, this command creates the log destination in the recipient account (222222222222) in us-east-1:
>aws logs put-destination \ --destination-name "testDestination" \ --target-arn "arn:aws:kinesis:us-west-2:222222222222:stream/YourStreamName" \ --role-arn "arn:aws:iam::222222222222:role/YourIAMRoleName" --region us-east-1
2. Create a subscription filter in your account.
3. (Optional) Check that your data stream is working by validating the flow of log events.