Follow

S3 - Known Issue - Amazon CloudWatch log data missing or invalid

Solutions:

This issue typically occurs because the time interval for which you want to retrieve log data must be expressed using timestamps expressed in terms of milliseconds elapsed since Jan 1, 1970 00:00:00 UTC. This can cause problems for end that routinely measure time in terms of nanoseconds, which is the default time interval used with the Bash command shell for Linux.

You retrieve CloudWatch log data from Amazon S3 by specifying the interval of time for which you wish to retrieve log data with starting and ending timestamps expressed in milliseconds. As the Exporting Log Data in Bulk to Amazon S3 "Concepts" section states:

from (timestamp)

A required timestamp expressed as the number of milliseconds since Jan 1, 1970 00:00:00 UTC. All log events in the log group that were ingested after this time will be exported.

to (timestamp)

A required timestamp expressed as the number of milliseconds since Jan 1, 1970 00:00:00 UTC. All log events in the log group that were ingested before this time will be exported.

If you need to retrieve CloudWatch log data exported to an Amazon S3 bucket or folder for the preceding two-hour period, you could use the following syntax (note the --from and --to parameters):

aws logs create-export-task --task-name "example-task" --log-group-name "/var/logs/example-logs" --from $(($(date -d "-2 hours" +%s%N)/1000000)) --to $(($(date +%s%N)/1000000)) --destination " log_bucket" --destination-prefix "example-logs"

 

If you continue to experience difficulties, please open a case with the DLT Opscenter using the following contact information:

1-888-358-7658

Opscenter@dlt.com

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk