kibanalogstash-grok

How to parse this content in kibana using grok pattern?

The message in kibana is:

{"log":"2024-02-01 10:30:00.004  INFO 1 --- [pool-1-thread-2] c.ankon.timer.ReminderExecCheckSchedule  : Detecting Timed ，start...\n","stream":"stdout","time":"2024-02-01T02:30:00.01291984Z"}

I'm trying to write a grok expression to do the matching:

%{QS:LOGTYPE}:%{QS:CONTENT},%{QS:ST}:%{QS:TIME}:%{QS:timestamp}

But it's not right

The question 1:

I'd like to ask how the grok expression should be written to match the above.

The question 2:

If I want the end result to be:

{"log":"2024-02-01 10:30:00.004  INFO 1 --- [pool-1-thread-2] c.ankon.timer.ReminderExecCheckSchedule  : Detecting Timed ，start...\n"}

What is the content of the grok pattern?

Solution

  • You can use built-in Grok Debugger in Kibana. http://localhost:5601/app/dev_tools#/grokdebugger

    Input

    {"log":"2024-02-01 10:30:00.004  INFO 1 --- [pool-1-thread-2] c.ankon.timer.ReminderExecCheckSchedule  : Detecting Timed ，start...\n","stream":"stdout","time":"2024-02-01T02:30:00.01291984Z"}

    output:

    {
  "loglevel": "INFO",
  "thread": "1",
  "message": "Detecting Timed ，start...\\n\",\"stream\":\"stdout\",\"time\":\"2024-02-01T02:30:00.01291984Z\"}",
  "class": "c.ankon.timer.ReminderExecCheckSchedule ",
  "timestamp": "2024-02-01 10:30:00.004",
  "threadname": "pool-1-thread-2"
}

    You can use the following link to see all grok patterns. https://github.com/hpcugent/logstash-patterns/blob/master/files/grok-patterns

    The question 2: If you want the end result like the following:

    {"log":"2024-02-01 10:30:00.004  INFO 1 --- [pool-1-thread-2] c.ankon.timer.ReminderExecCheckSchedule  : Detecting Timed ，start...\n"}

    you can use another grok like this:

    %{DATA:raw_data}\\n

    The above grok pattern will parse the data and only take the part before \n

    enter image description here