grafana-lokipromtail

Promtail: How to remove timestamps from filenames?

I have a simple problem:

My logfiles have timestamps in their name, i.e.:

/var/log/html/access-2021-11-27.log
/var/log/html/access-2021-11-28.log
/var/log/html/access-2021-11-29.log

Promtail is scraping this but does not "see" that access-2021-11-28.log is a continuation of access-2021-11-27.log. So it will "detect" a log file access-2021-11-28.log on the 28th and not show the access-2021-11-27.log anymore. I would want to see just "access.log" with data for several days.

I would assume this should be a well-known scenario, but I cannot find anything on this on the Internet.

Solution

  • The only way is to change log configuration of the application which is generating the logs, to use a unique access.log instead of the schema of the access-xxxx-xx-xx.log files. Unfortunately, this is not always possible.

    But...

    The old files can still be shown, it only depends on the time range used. Here is an example:

    enter image description here

    You can use regular expressions to perform the query, like in this example:

    {filename=~".*JIRA_INSTALL/logs/access_log\\..*"}