[SOLVED] Can filebeat dissect a log line with spaces?

Can filebeat dissect a log line with spaces?

So I have a log line formatted as such:

2020-04-15 12:16:44,936 WARN c.e.d.c.p.p.BasePooledObjectFactory [main] Caution - XML schema validation has been disabled! Validation is only available when using XML.

I am using filebeat to send this directly to elasticsearch, which it does but the log.level is not set, the whole line becomes the message.

reading up on dissection I had intended to use:

processors:
  - add_host_metadata: ~
  - dissect:
     tokenizer: "%{} %{} %{log.level} %{} [%{}] %{message}"
     field: "message"
     target_prefix: ""

which I expected to split into:

{ 
  log.level: WARN
  message: Caution - XML schema validation has been disabled! Validation is only available when using XML.
}

instead I get the same output as without the dissect:

{
  message: 2020-04-15 12:16:44,936 WARN c.e.d.c.p.p.BasePooledObjectFactory [main] Caution - XML schema validation has been disabled! Validation is only available when using XML.
}

I'm just getting to grips with filebeat and I've tried looking through the documentation which made it look simple enough. however my dissect is currently not doing anything. host metadata is being added so I believe that the processors are being called.

How can I get the log level out of the log line? (preferably without changing the format of the log itself)

Solution

You need to pick another field name than message in the dissect tokenization since this is the name of the field that contains the original log message:

processors:
  - add_host_metadata: ~
  - dissect:
     tokenizer: "%{} %{} %{log.level} %{} [%{}] %{msg}"
     field: "message"
     target_prefix: ""