I have configured Amazon CloudWatch to export log data to Amazon S3 as described at Exporting Log Data to Amazon S3 Using the AWS CLI. After running the AWS CLI command to configure Amazon CloudWatch to export log data, I receive a response indicating that the command completed successfully. The AWS CLI command I used had the following syntax:

aws logs create-export-task --task-name "example-task" --log-group-name "/var/logs/example-logs" --from 1442880000 --to 1447891200 --destination "log_bucket" --destination-prefix "example-logs"

Amazon CloudWatch responded to my command with a taskID value similar to the value in the following response message, indicating that the command completed successfully:

{

    "taskId": "05d83eb7-272b-4cd1-8d9d-e3346e790483"

}

Unfortunately, I am unable to locate any log file data in S3.

This issue typically occurs because the time interval for which you want to retrieve log data must be expressed using timestamps expressed in terms of milliseconds elapsed since Jan 1, 1970 00:00:00 UTC. This can cause problems for end that routinely measure time in terms of nanoseconds, which is the default time interval used with the Bash command shell for Linux.

You retrieve CloudWatch log data from Amazon S3 by specifying the interval of time for which you wish to retrieve log data with starting and ending timestamps expressed in milliseconds. As the Exporting Log Data in Bulk to Amazon S3 "Concepts" section states:

from (timestamp)

A required timestamp expressed as the number of milliseconds since Jan 1, 1970 00:00:00 UTC. All log events in the log group that were ingested after this time will be exported.

to (timestamp)

A required timestamp expressed as the number of milliseconds since Jan 1, 1970 00:00:00 UTC. All log events in the log group that were ingested before this time will be exported.

If you need to retrieve CloudWatch log data exported to an Amazon S3 bucket or folder for the preceding two-hour period, you could use the following syntax (note the --from and --to parameters):

aws logs create-export-task --task-name "example-task" --log-group-name "/var/logs/example-logs" --from $(($(date -d "-2 hours" +%s%N)/1000000)) --to $(($(date +%s%N)/1000000)) --destination " log_bucket" --destination-prefix "example-logs"

Amazon CloudWatch, export Log data, Amazon S3, timestamp, Linux, millisecond, create-export-task


Did this page help you? Yes | No

Back to the AWS Support Knowledge Center

Need help? Visit the AWS Support Center

Published: 2016-02-30