In my scenario, I'm using a fluent-bit (version 1.6.1) filter to identify a particular log pattern and tag (from.solr.out) it.
[FILTER]
Name rewrite_tag
Match test_tag
Rule $msg "^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.*$" from.solr.out false
Emitter_Name re_emitted
Then I modify the record to add a field like below
[FILTER]
Name record_modifier
Match from.solr.out
Record aid 53
Now I need to give a new tag (format.logging) to that modified record which I tried with below code snippet and it doesn't work.
[FILTER]
Name rewrite_tag
Match from.solr.out
Rule $from.solr.out "^.*$" format.logging false
Emitter_Name re_emitted_indexing
What is wrong with that approach?
Is there any better approach/filter to achieve this task?
Figure out how to get the result that I expected.
...
[2020/10/22 12:49:07] [ info] [sp] stream processor started
[0] format.logging: [1603351147.631883694, {"msg"=>"2020-10-16 09:29:03.968 WARN ...
I thought the original message that comes with the tag $msg is disappearing after adding the new tag (from.solr.out). Because in my second filter, I'm adding a record to the second tag (from.solr.out) not the initial tag ($msg).
Then in the last filter, I was trying to create a new tag (format.logging) by using the second tag (from.solr.out). That's where things have gone wrong. Instead of using the second tag, I tried to use the first tag ($msg) here. Then I got all the changes I needed as the output.
Changes I was expecting were;
The final change I've done is
[FILTER]
Name rewrite_tag
Match from.solr.out
Rule $msg "^.*$" format.logging false
Emitter_Name re_emitted_indexing