logginglogstashlog-shippingfluentd

Fluentd to Logstash output plugin


I am trying to read from the scribe server using flunetd and output those logs to be stored in logstash for now. I know it's very stupid to log the scribe_central logs to another central logger, but we need this to be done in our current architecture.

Does anyone know if there is any plugin to do that? I searched Google but could not find any.


Solution

  • For Scribe <-> Fluentd, there is fluent-plugin-scribe:

    For Fluentd <-> Logstash, a couple of options:

    1. Use Redis in the middle, and use fluent-plugin-redis and input_redis on Logstash's side. So it would be Fluentd -> Redis -> Logstash. This is what Logstash recommends anyway with log shippers + Logstash.
    2. Alternatively, you can use Fluentd's out_forward plugin with Logstash's TCP input. Logstash has Fluentd codec to handle the input coming from Fluentd.