I have multiple log files, thus my loki stream looks like:
{job="my_job", filename="20230920_1.log"}
{job="my_job", filename="20230920_2.log"}
I generate these files each day at 16pm. And in each file it should has 2 sentences simulate from and error=0
I use Loki and Grafana to monitor my logs. So now I want to config a alert. For example, each day at 17pm after these files are generated, the alert should check all files and see if any file does not has these 2 sentences.
I'm new to Loki and LogQL, and as far as I know LogQL is used to query each line in one stream. Anyway hope this is not a useless question.
Besides, if Loki does not fit in this problem, is there something else do? (VictoriaMetrics...?)
Let me rephrase your question according to my understanding. You periodically ingest new files with many log lines into Loki. Log lines per every file are ingested into a dedicated log stream, which is uniquely identified by job and filename labels. You want detecting log streams, which have no both simulate from and error=0 phrases across all the logs lines in the stream.
If my understanding is correct, then you need to use count_over_time() function for counting log lines containing at least one of the required phrases, and then wrap this function into sum(...) by (job, filename) != 2 (see docs for sum() funtion). Try the following LogQL query then, which should return log streams, which have no both required phrases over the logs with timestamps for the last day:
sum by (job, filename) (
count_over_time({job="my_job"} |~ "simulate from|error=0" [1d])
) != 2
Note that this query may return invalid log streams, which contain two lines with the simulate from phrase and don't contain lines with error=0 phrase, or vice versa. I don't know how to write LogQL query, which properly handles such cases :( It looks like it is impossible to do.
It is easier to write and understand this query with LogsQL in VictoriaLogs though:
_time:1d ("simulate from" or "error=0")
| stats by (_stream) count() as matching_lines
| filter matching_lines:=2
It is even possible to write LogsQL query, which is free from false positive issues mentioned above, by using join pipe:
_time:1d "simulate from" | by (_stream) count() x
| join by (_stream) (_time:1d "error=0" | by (_stream) count() y)
| filter x:=1 y:=1
Disclaimer: I'm the core developer of VictoriaLogs.