My kubernetes have liveness enable, and it log on application, like this:
kubectl logs -n example-namespace example-app node-app
::ffff:127.0.0.1 - - [17/Sep/2020:14:12:19 +0000] "GET /docs HTTP/1.1" 301 175
::ffff:127.0.0.1 - - [17/Sep/2020:14:13:19 +0000] "GET /docs/ HTTP/1.1" 200 3104
::192.168.0.1 - - [17/Sep/2020:14:13:19 +0000] "GET /home-page HTTP/1.1" 200 3104
::ffff:127.0.0.1 - - [17/Sep/2020:14:13:19 +0000] "GET /docs HTTP/1.1" 301 175
::ffff:127.0.0.1 - - [17/Sep/2020:14:13:22 +0000] "GET /docs/ HTTP/1.1" 200 3104
I Use fluentD to send logs to ClowdWatch. My fluentD configuration:
How can i filter, to fluentD only matches
::192.168.0.1 - - [17/Sep/2020:14:13:19 +0000] "GET /home-page HTTP/1.1" 200 3104
And ignore
::ffff:127.0.0.1 - - [17/Sep/2020:14:13:19 +0000] "GET /docs HTTP/1.1" 301 175
Thanks!
After some research, i found this solution:
<match kubernetes.var.log.containers.**_kube-system_**>
@type null
</match>
and this
<filter **>
@type grep
exclude1 log docs
</filter>
The reference:
https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter/issues/91
https://docs.fluentd.org/filter/grep
EDIT
or add:
exclude_path ["/var/log/containers/cloudwatch-agent*", "/var/log/containers/fluentd*", "/var/log/containers/*istio*"]
this config ignore the source files with pattern istio.