I am new to all this architecture, and I am a bit lost.
I have integrated all these instruments in my Rails application:
gem "opentelemetry-exporter-otlp"
gem "opentelemetry-instrumentation-mysql2"
gem "opentelemetry-instrumentation-net_http"
gem "opentelemetry-instrumentation-rack"
gem "opentelemetry-instrumentation-rails"
I am sending the metrics to an instance of OpenTelemetryCollector, and from there to a Tempo instance.
In Grafana, I have configured the Tempo datasource.
I can see the Traces in the Grafana > Explorer:
But I don't know how to build a proper dashboard with it. I have tried to import a dashboard like this. But the data source is Prometheus, not Loki. The are only 2 dashboards for Tempo on this database. And they don't look like the one I want.
I would like to see the requests, the statuses, the request_time, ... as in the OpenTelemetry APM Dashboard
Any help?
If my architecture is wrong, I am also willing to change it.
The solution came from one of the Grafana community members:
See doc for OpenTelemetry APM Dashboard
- OpenTelemetry Collector: Receive traces and process them to metrics with Span Metrics Connector
- Prometheus: Scrape metrics from OpenTelemetry Collector
You are missing these steps (older collector versions have had spanmetricsprocessor, not connector) - this connector (processor) aggregates Request, Error and Duration (R.E.D) metrics from span data - so you will generate METRICS from TRACES and you will save these generated METRICS into metric storage - Prometheus.