elasticsearchgrafanagrafana-variablegrafana-templatinggrafana-elasticsearch

Grafana - Elasticsearch dashboard variable does not provide any results

I am trying to set up a logs monitoring where I can filter logs by log.level and labels.Application fields from a .NET application which uses a Serilog sink for Elastic.

I have created a dashboard variable and defined some JSON to query for labels.Application, but I never get any response and I do not know why. Below is an example where I tried to query for host.name:

query

Now, when I go to my Elasticsearch datasource and I click on Explore, if I filter for logs, I am actually able to see both host.name and labels.Application.

working example

I do not know why this works everywhere else but it somehow does not work in dashboard variables. Is there something that would need to be explicitly set up for this to work? All the examples

Data source configuration (though I would imagine it’s got nothing to do with it):

datasource config

Any idea what could be behind this? I am out of ideas, to be honest.

Solution

  • As it turns out (after an additional hour of debugging and searching), I had a provisioned datasources/default.yml where the pre-configured Elasticsearch datasource had the interval set to Daily. After commenting it out, it works fine now. I have no idea why this was necessary, but perhaps worth pointing out.

    apiVersion: 1

datasources:
- name: Elasticsearch
  type: elasticsearch
  uid: elasticsearch
  access: proxy
  url: http://elasticsearch:9200
  jsonData:
    index: 'logs-*'
    # interval: Daily
    timeField: '@timestamp'