logstashlogstash-grok

How to match a data that appears 0 or 1 times in grok?

I appreciate if someone can help me out with the logstash frok.

I am trying to create a single grok definition for below samples. My goal is to parse the subject as a field.

  1. t1='t1', t2='t2', subject='subj', t3='hoge'
  2. t1='t1', subject='subj', t3='hoge'
  3. t1='t1', t3='hoge'

Below works fine if subject always exists.

t1=%{QS}, (%{NOTSPACE:key}=%{NOTSPACE:value}, )*subject=%{QS:subject}

However, I will need to deal with sample No.3 , so if I change the grok definition as below, then sample No1. and No2. is not working now.

t1=%{QS}, (%{NOTSPACE:key}=%{NOTSPACE:value}, )*(subject=%{QS:subject})?

Is there any good grok definition that can work with all samples?

Solution

  • your best bet is probably to just conditionally parse the subject out:

    if [message] =~ /subject/ {
  grok {
    match => { "message" => "subject=%{QS:subject}" }
  }
}

    you can still do your unconditional grok as well and all matches will be added to the event.