[SOLVED] Grok pattern for matching content in already parsed log line

Grok pattern for matching content in already parsed log line

I will try to explain what I want to achieve. I have grok pattern to match simple log line. There is response message json body in the log line and I am able to parse it with custom regex pattern and I see it in kibana dashboard as expected. The thing is I would like to extract some data from the body itself.

Here is the working grok pattern:

{TIMESTAMP_ISO8601:timestamp}%{SPACE}*%{LOGLEVEL:level}:%{SPACE}*%{DATA}%{BODY:body}

Custom pattern for BODY:

BODY (Body:.* \{.*})

And I see it parsed using grok debugger:

{
   "timestamp": "2022-11-04 17:09:28.052",
   "level": "INFO",
   "body": {\"status\":200,\"page\":1, \"fieldToBeParsed\":12 .....//more json conten}
}

Is there any way to parse some of the content of the body together with the whole body. So I can get result similar to:

{
    "timestamp": "2022-11-04 17:09:28.052",
    "level": "INFO",
    "body": {\"status\":200,\"page\":1, \"fieldToBeParsed\":12 .....//more json conten},
    "parsedFromBody: 12
}

Example:

{TIMESTAMP_ISO8601:timestamp}%{SPACE}*%{LOGLEVEL:level}:%{SPACE}*%{DATA}%{BODY:body}% 
{FROMBODY:frombody}%

Thank you!

Solution

Once you have used grok to parse a field, you can use a second grok filter to parse fields created by the first grok. Do not try to do both in one grok, it may or may not work. The matches are a hash, and Java hashes are not ordered.

grok { match => { "body" => "fieldToBeParsed\":%{NUMBER:someField:int}" } }