elasticsearchelastic-stackopen-telemetry

OpenTelemetry Export to Elastic Search without Elastic APM

I'd like to export trace/metrics data to Elastic Search using OpenTelemetry, but I'd prefer to avoid Elastic APM. Is it possible? The opentelemetry contrib repo apparently suggests it is possible, however, I did not find anything on elastic.co documentation. By the way, openapm.io implies see here, OpenTelemetry can export to elastic beats (which is extremely desirable), but again, I did not find anything in Elastic.co docs.

Solution

  • After trying this for some time, I've finally made it to work with the latest version of opentelemetry-collector-contrib (0.60.0 at the time of this writting).

    First I've created a multi-node cluster with Docker Compose using the steps here https://www.elastic.co/guide/en/elasticsearch/reference/8.4/docker.html#docker-compose-file. I also had to follow the instructions to set the vm.max_map_count to at least 262144.

    Then, using the same net, I started an opentelemetry-collector-contrib docker image using a command like:

    docker run --net certs_default --name opentelemetry-collector -v C:\docker\elastic\otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml -p 4317:4317 -p 4318:4318 otel/opentelemetry-collector-contrib:0.60.0

    The configuration file is very simple:

    receivers:
  otlp:
    protocols:
      grpc:
      http:

exporters:
  logging:
    loglevel: debug

  elasticsearch/trace:
    endpoints: [https://es01:9200]
    user: elastic
    password: 
    api_key: 
    tls:
      insecure_skip_verify: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [elasticsearch/trace]

    Finally, I've just configured my application to use the OtlpExporter:

    .AddOtlpExporter(configure =>
{
    configure.Protocol = OpenTelemetry.Exporter.OtlpExportProtocol.Grpc;
})

    Doing these steps, I can see my traces in Kibana. Now it would be nice to have some template to display a single trace with all the spans.