azure-log-analyticsazure-sentinel

TimeGenerated field not taking provided date


I am writting custom logs to log analytics.

Based on the following link:

https://learn.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api#create-a-request

I should be able to use a field in the request header called: time-generated-field, the documentation says that: " If you specify a field, its contents are used for TimeGenerated. If you don't specify this field, the default for TimeGenerated is the time that the message is ingested. The contents of the message field should follow the ISO 8601 format YYYY-MM-DDThh:mm:ssZ"

I am passing the following value: 2021-11-11T19:52:45Z (as a string, since you can't pass this as a datetime object) but the problem is that when I look in the log analytics workspace, the TimeGenerated field is this (today's date): 2021-12-01T18:41:04.529Z which is the datetime the event is ingested, so basically, it's not taking the real event generated time which is 2021-11-11T19:52:45Z which I am passing in the header.

Am I doing something wrong here?

Any help would be appreciated, I am running out of ideas here.


Solution

  • It turns out I misread the documentation and didn't realize that the time-generated-field is not an argument you pass in, but a property in the body of the JSON data that you are sending to the log analytics workspace.

    It can be referenced in this manner (this is an example of 1 way of doing it):

    def post_data(customer_id, shared_key, body, log_type):
        method = 'POST'
        #The string "raised" assigned to the TimeStampField variable below 
        #is an actual property in the JSON object that I am sending to log 
        #analytics, it contains the datetime in the expected format.
        TimeStampField = "raised" 
        content_type = 'application/json'
        resource = '/api/logs'
        rfc1123date = datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')
        content_length = len(body)
        signature = build_signature(customer_id, shared_key, rfc1123date, content_length, method, content_type, resource)
        uri = 'https://' + customer_id + '.ods.opinsights.azure.com' + resource + '?api-version=2016-04-01'
        headers = {
            'content-type': content_type,
            'Authorization': signature,
            'Log-Type': log_type,
            'x-ms-date': rfc1123date,
            'time-generated-field': TimeStampField
        }
    

    Will definitely create a pull request to modify the wording here: https://learn.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api#create-a-request As I feel it is not very intuitive