I'm trying to setup some processors in a filebeat.yml to process some logs before sending to ELK.
An important part of the processing is determining the "level" of the event, which is not always included in the line in the log file.
This is the idea I have for it right now:
# /var/log/messages
- type: log
processors:
- dissect:
tokenizer: "%{month} %{day} %{time} %{hostname} %{service}: {%message}"
field: "message"
target_prefix: "dissect"
- if:
when:
regexp:
message: ((E|e)rror|(f|F)ault)
then:
- add_fields:
target: 'dissect'
fields:
level: error
else:
- if:
when:
regexp:
message: (W|W)arning
then:
- add_fields:
target: 'dissect'
fields:
level: warning
else:
- add_fields:
target: 'dissect'
fields:
level: information
- drop_fields:
#duplicate
fields: ["dissect.month","dissect.day","dissect.time","dissect.hostname","message"]
# Change to true to enable this input configuration.
enabled: true
paths:
- /var/log/messages
I'm still not sure about those patterns I'm trying... but right now I don't think they're what's causing me to fail.
When trying to run filebeat with console output for a test with
filebeat -e -c filebeat.yml
I get the following error:
2022-01-26T17:45:27.174+0200 ERROR instance/beat.go:877 Exiting: Error while initializing input: failed to make if/then/else processor: missing or invalid condition
Exiting: Error while initializing input: failed to make if/then/else processor: missing or invalid condition
I'm very new to yaml in general, and the only other beat I've done before is an AuditBeat (which works, and has conditions, but not "if"s). Does anyone know what the problem might be?
To clarify: I commented out all other "input" entries, leaving just this one, and still got this error.
Edit: Version: 7.2.0
The if
part of the if-then-else
processor doesn't use the when
label to introduce the condition. The correct usage is:
- if:
regexp:
message: [...]
You have to correct the two if
processors in your configuration.
Additionally, there's a mistake in your dissect expression. {%message}
should be %{message}
. Also, the regexp for warning should be (W|w)arning
not (W|W)arning
(both W's are uppercase in your config).
This is the corrected processors configuration:
processors:
- dissect:
tokenizer: "%{month} %{day} %{time} %{hostname} %{service}: %{message}"
field: "message"
target_prefix: "dissect"
- if:
regexp:
message: ((E|e)rror|(f|F)ault)
then:
- add_fields:
target: 'dissect'
fields:
level: error
else:
- if:
regexp:
message: (W|w)arning
then:
- add_fields:
target: 'dissect'
fields:
level: warning
else:
- add_fields:
target: 'dissect'
fields:
level: information