[SOLVED] Perform avg_over_time grouping by a field which is modified in grafana

Perform avg_over_time grouping by a field which is modified in grafana

I have logs like so in loki which I query on grafana:

{
   "alpha" : 10,
   "log_response_time" : 0.23,
   "log_request" : "/core/login?abc=txt
}

Some log_request fields may have URL parameter some may not. I want to find the AVG log_response_time for the above logs grouped by log_request, where log_request without the URL parameters is considered. The reason to do so is otherwise the result set ends up having >500 unique values, and it fails. Besides it makes no sense to consider similar API separately.

So I have the following query:

avg_over_time({container="/web", hostname="abc-prod"} | json | line_format `{{ __line__  | replace .log_request (regexReplaceAll "\\?.*$" .log_request "")  }}` | unwrap log_response_time [1m]) by (log_request)

However, it turns out the group by considers the original logs, and not the modified one.

I confirmed that the following works well to remove the URL parameters as expected, and the log_request is devoid of the url parameters:

{container="/web", hostname="abc-prod"} | json | line_format `{{ __line__  | replace .log_request (regexReplaceAll "\\?.*$" .log_request "")  }}`

How can I query grafana to make it group by the log_request without the URL parameters?

Solution

| line_format doesn't rewrite any of the labels associated with log item, it only rewrites a message (basically the line shown in log panel when item is collapsed).

Since before this call you already parsed labels with | json, not labels are affected.

You can use | label_format instead, to replace content of the label. I believe you need a query like this:

{container="/web", hostname="abc-prod"}
  | json
  | label_format log_request=`{{regexReplaceAll "\\?.*" .log_request "" }}`

It will delete everything starting with first ? from label log_request