open-telemetrygrafana-lokifluent-bit

FluentBit: Adding OpenTelemetry attributes to forwarded log (target: Loki)


I have a FluentBit container receiving data from a WebAPI via OpenTelemetry input. Using stdout output I see, that the log is added to FluentBit in the structured form (including attributes/properties/...whatever-you-want-to-call-it...), not only the pure message.

e.g.:

[2] opentelemetry.0: [[1717162605.894517831, {"EnvName"=>"Development", "{OriginalFormat}"=>"Hosting environment: {EnvName}"}], {"message"=>"Hosting environment: Development"}]

Not the EnvName in the first [[. Now I want that output to Loki target. The fluent bit YAML I use:

pipeline:
    inputs:
        - name: opentelemetry
          listen: 0.0.0.0

    outputs:
        - name: loki
          match: '*'
          host: host.docker.internal
          port: 3100
          labels: job=my

Loki is a different container on same host. The connection itself does work, but Loki only receives the last part, the {"message"=>"Hosting environment: Development"} without the attributes. But I do know Loki is able to handle structured logs, I did it in a different project (without OTel in between).

Now the question is: How do I tell FluentBit to include the attributes it receives from OTel in the message it sends to Loki?

(Bonus point if someone can tell me which tags I need to use in match for Loki output. Neither v1_logs nor opentelemetry.proto.collector.log.v1.LogService_Export works. I had to use '*' to get anything matching. API sends OTel via gRPc)


Solution

  • For the record: It's not currently possible, but was considered an oversight by FluentBit project: https://github.com/fluent/fluent-bit/issues/8910

    At the time of this writing it was planned for Fluent Bit v3.1.0 milestone (see GitHub ticket).