Why can't I find my DataSync task logs in the CloudWatch log group?

Last updated: 2020-12-14

I ran an AWS DataSync task, but I can't find the logs in the relevant Amazon CloudWatch log group. How can I troubleshoot this?

Resolution

Confirm that CloudWatch Logs has a resource policy that allows DataSync to upload logs.

Follow these steps to review the CloudWatch Logs resource policies in the AWS Region of your DataSync agent:

1.    Run the describe-resource-policies command using the AWS Command Line Interface (AWS CLI):

aws logs describe-resource-policies --region us-east-1

Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.

2.    Review the output of the command. If a resource policy isn't set up, then the output is similar to the following:

{
    "resourcePolicies": []
}

Important: Confirm that the resource policy for DataSync is enabled in the correct AWS Region. The policy must be in the same Region as the DataSync agent that you're using.

Follow these steps to create a resource policy that grants DataSync permissions for uploading logs:

1.    Create a JSON file that grants DataSync the minimum permissions for uploading logs:

{
    "Statement": [
       {
            "Sid": "DataSyncLogsToCloudWatchLogs",
            "Effect": "Allow",
            "Action": [
                "logs:PutLogEvents",
                "logs:CreateLogStream"
            ],
            "Principal": {
                "Service": "datasync.amazonaws.com"
            },
            "Resource": "*"
        }
    ],
    "Version": "2012-10-17"
}

You can name the file policy.json.

2.    Run the put-resource-policy command using the AWS CLI to create a resource policy using the JSON file:

aws logs put-resource-policy --policy-name trustDataSync --policy-document file://policy.json --region <Region>

Important: Set the AWS Region of your DataSync agent as the value for --region.

3.    Run the describe-resource-policies command to confirm that the resource policy was created:

aws logs describe-resource-policies --region <Region>

4.    After you create the resource policy, the command output is similar to the following:

{
    "resourcePolicies": [
        {
            "policyName": "trustDataSync",
            "policyDocument": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"DataSyncLogsToCloudWatchLogs\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"datasync.amazonaws.com\"},\"Action\":[\"logs:PutLogEvents\",\"logs:CreateLogStream\"],\"Resource\":\"*\"}]}",
            "lastUpdatedTime": 1577448776606
        }
    ]
}

5.    To test the resource policy, run a DataSync task. A few minutes after the task runs, confirm that you're seeing the log stream from the relevant CloudWatch log group.


Did this article help?


Do you need billing or technical support?