logstashlogstash-grok

Unable to use grok in Logstash pipeline to manipulate message


I am trying to use grok in our Logstash pipelines. I would rather not share my entire file, but I am confident that the rest of the file is correct as we use it in our other pipelines.

What I have in my Logstash pipeline:

    {
       patterns_dir => "..\config\pattern"
       match => { "message" => "%{TIMESTAMP_PROMATIC:promatic_timestamp}%{SPACE}\[%{LOGLEVEL_SERILOG:log.level}\]%{SPACE}%{GREEDYDATA:message}" }
    }
    date 
    {
        match => ["promatic_timestamp", "HH:mm:ss.SSS"]
        remove_field => [ "promatic_timestamp" ]
    }

How the logging looks like:

15:05:05.303 [DBG] some random text file

What it now looks like: enter image description here

What it should look like: ![enter image description here

In other words, time stamp can be removed and the log level be assigned to the variable: log.level

Thanks in advance!


Solution

  • This config solved my problem:

    grok 
    {
        patterns_dir => "..\config\pattern"
        match => { "message" => "%{TIMESTAMP_PROMATIC:promatic_timestamp} \[%{LOGLEVEL_SERILOG:log.level}\] %{GREEDYDATA:message}" }
        overwrite => [ "message" ]
    }
    date 
    {
        match => ["promatic_timestamp", "yyyy MM dd HH:mm:ss.SSS"]
        remove_field => [ "promatic_timestamp" ]
    }