prometheusgrafanagrafana-lokigrafana-api

Send logs directly to Loki without use of agents

Is there a way to send logs to Loki directly without having to use one of it's agents?

For example, if I have an API, is it possible to send request/response logs directly to Loki from an API, without the interference of, for example, Promtail?

Solution

  • Loki HTTP API

    Loki HTTP API allows pushing messages directly to Grafana Loki server:

    POST /loki/api/v1/push

    /loki/api/v1/push is the endpoint used to send log entries to Loki. The default behavior is for the POST body to be a snappy-compressed protobuf message:

    Alternatively, if the Content-Type header is set to application/json, a JSON post body can be sent in the following format:

    {
  "streams": [
    {
      "stream": {
        "label": "value"
      },
      "values": [
          [ "<unix epoch in nanoseconds>", "<log line>" ],
          [ "<unix epoch in nanoseconds>", "<log line>" ]
      ]
    }
  ]
}

    You can set Content-Encoding: gzip request header and post gzipped JSON.

    Example:

    curl -v -H "Content-Type: application/json" -XPOST -s "http://localhost:3100/loki/api/v1/push" --data-raw \
 '{"streams": [{ "stream": { "foo": "bar2" }, "values": [ [ "1570818238000000000", "fizzbuzz" ] ] }]}'

    So it is easy to create JSON-formatted string with logs and send it to the Grafana Loki.

    Libraries

    There are some libraries implementing several Grafana Loki protocols.

    There is also (my) zero-dependency library in pure Java 1.8, which implements pushing logs in JSON format to Grafana Loki. Works on Java SE and Android platform:

    Security

    Above API doesn't support any access restrictions as written here - when using over public network, consider e.g. configuring Nginx proxy with HTTPS from Certbot and Basic Authentication.