tcpproxylogstashlogz.io

Logstash: forwardings logs via proxy to logz.io


I want to ship stdout from a running application to logz.io using logstash. Application and logstash are both docker images managed by docker-compose, which does the setup (pull images, network_mode, logging driver etc.). Logstash input is handled via gelf-input-plugin. The shipping to logz.io is handled via tcp-output-plugin.

logstash.conf:

input { gelf {
            type => docker
            port => 12201 } }

filter { mutate { add_field => { "token" => "${LOGZIOTOKEN}" } } }

output { tcp {
            host => "listener.logz.io"
            port => 5050
            codec => json_lines } }

excerpt from docker-compose.yml:

  application:
  ...
 logging:
   driver: "gelf"
  options:
    gelf-address: "udp://0.0.0.0:12201"

This works as expected.

Now there is a TCP proxy server I need to use, to ship the logs from the host (running the logstash instance) to logz.io. Unfortunately I did not find a proxy extension for logstashs tcp-output-plugin. Does anyone has a suggestion for this issue?


Solution

  • The logstash's http output plugin has the attribute proxy. You have to use the logz.io port for shipping with curl: 8070(http)/8071(https).

    A working config looks like this:

    output { http {
            url => "https://listener.logz.io:8071?token=${LOGZIOTOKEN}"
            http_method => "post"
            format => "json"
            content_type => "application/json"
            proxy => {
                host => "${PROXYHOST}"
                port => "${PROXYPORT}"
                scheme => 'http'
                user => "${PROXYUSER}"
                password => "${PROXYPW}"
            }}}
    

    You do not need the filter to ship to logz.io like in tcp-output-plugin config. Just add the input and ship it!