fluentfluent-bit

Fluentbit Regex support in Match for Filter plugins


I have a basic question about the usage of Match for Filter plugins. I saw that Match supports wildcard, based on https://docs.fluentbit.io/manual/concepts/data-pipeline/router Is it only basic wildcard support or complete regex support? My typical tags look like part1.part2.part3.part4

This is my current configuration and it works as expected (i.e. logs that match these tags are getting throttled).

    [FILTER]
        Name                      throttle
        Match                     part1.*.*.part4
        Rate                      1
        Window                    300
        Interval                  1s

I tried to modify this so that this filter is not applied if part2=xyz

    [FILTER]
        Name                      throttle
        Match                     part1.(?!xyz).*.part4
        Rate                      1
        Window                    300
        Interval                  1s

But looks like this isn't working (i.e. none of the logs are getting throttled)

My questions:

  1. Does it only support basic wildcard support or has complete regex support?
  2. Is there a way to do what I am trying to do in the throttle plugin?

Solution

  • Found the answer

    TIL: there is a key named Match_Regex and it works in all the places where Match can be used. Match only support * as a wildcard Match_Regex supports whole regex

    Source: https://docs.fluentbit.io/manual/v/1.3/configuration/file

    My modified config

        [FILTER]
            Name                      throttle
            Match_Regex               part1.(?!xyz).*.part4
            Rate                      1
            Window                    300
            Interval                  1s
    

    Thanks to Anurag Gupta for pointing to this in Fluentbit slack.