I am using this code part in my promtail-config.yaml
file to filter the log lines that contains the word INFO
.
pipeline_stages:
- match:
selector: '{env="myenv"}'
stages:
- json:
expressions:
msg: log
- match:
selector: '{status="info"} !~ ".*INFO.*"'
action: drop
- output:
source: msg
But the problem is when I use this approach to filter the logs it cause to remove the stack trace
of INFO
logs since the stack trace
consists of couple of separate lines, Promtail consider it as a separate log line and that log line removed in the filtering process as that line not containing the word INFO
in it.
How to keep the stack trace when carry on this filtering process?
Any help would be greatly appreciated.
I resolved this by using multiline
under pipeline stages.
pipeline_stages:
- multiline:
firstline: '^\[\d{4}-\d{2}-\d{2} \d{1,2}:\d{2}:\d{2}\]'
max_wait_time: 3s
That makes the whole stack trace a one line and that resolves the issue.
For more details you can refer to the grafana documentation about this.
Furthermore, I highly encourage everyone who has problems regarding how to properly configure the promtail
, to refer the grafana documentation about promtail querying.