dockerkubernetessyslog

Send kubernetes container application logs (stdout and stderr) to UDP server

I know docker has options like syslog log-driver and log-opts so that it can be used to send the logs to say, the UDP server.

Marathon is the docker orchestrator here and a config file has the below:

    {
      "key": "log-driver",
      "value": "syslog"
    },
    {
      "key": "log-opt",
      "value": "syslog-address=udp://some-udp-server:port"
    },

The existing setup is such that certain downstream systems/entities take the information received on this UDP server to create visualisations on Grafana.

How do I achieve the same in a k8s manifest file that I'm deploying via helm3? Or is there a third-party application I need to use? Basically, I want to send the logs that come in the kubectl logs -f <pod_name> command to this UDP server with minimal intrusion. I would only like to replace this part of the flow so that I don't have to disturb any of the downstream systems.

Solution

  • As David suggested there is no option to control the log target. However as requested for log collector application writing this answer.

    If your application is streaming the UDP logs you can use the Graylog Opensource. It uses Mongo & Elasticsearch as backend databases. We been using Graylog to collect logs from the application POD.

    Now regarding the log collector for kubectl logs -f <POD> you can push all these logs from the Worker Node file system using the fluentd collector. Log location will be /var/log/pods

    You can use the Fluentd collector along with the Graylog Gelf UDP input

    Fluentd -> pushing over gelf UDP -> Graylog input saving to Elasticsearch

    Here is the ref you can follow : https://docs.fluentd.org/how-to-guides/graylog2

    Above example uses Graylog2 now Graylog3 version is available opensource would suggest checking out that.

    You can refer my Github repo : https://github.com/harsh4870/OCI-public-logging-uma-agent

    Will get more idea about how deployment setting up log file on Node's filesystem and further it gets processed by collector although not using fluentd but just for ref.

    Oracle OCI UMA agent also similar job like fluentd collector only, parsing & pushing logs to the backend.