parsing a tail file to process commands

I have found some info on this, but not enough to get me going with this project. I have limited bash experience, trying to learn.

I have a log file that I would like to read (tail) and process lines as they come in based on timestamp in the log.

Log example: from a tail -f filename.txt | grep --line-buffered RXKEY,MAIN:

20230913200618,12345,RXKEY,MAIN
20230913200620,12345,RXKEY,MAIN
20230913200627,12345,RXKEY,MAIN
20230913200629,12345,RXKEY,MAIN

(Timestamp,id,type,location)

I would like to read this file and perform an action, if the last entry was X amount of seconds before the previous entry. IE, if 2 logs hit in 4 or 5 seconds apart, run a command locally. Then keep monitoring for future logs and repeat as needed.

I have the log working but not sure how to read the timestamps and then process an action.

Solution

You can pipe grep into while read, which lets you loop over each line received.

#!/usr/bin/env bash

declare allow_difference=5
declare last=

# 'while read -r' to prevent backslashes from being interpreted
# https://unix.stackexchange.com/questions/192786/what-is-the-meaning-of-read-r
tail -f filename.txt | grep --line-buffered RXKEY,MAIN | while read -r line; do
    # read into array 'linearray', splitting on ',' by setting
    # the IFS (how bash chooses to split word boundaries)
    IFS=',' read -ra linearray <<< "$line"
    # [[ -z "$somevariable" ]] checks if the variable is empty
    # if its empty, then we havent set a last time yet, so set it and continue the while loop
    if [[ -z "$last" ]]; then
        last="${linearray[0]}"
        continue
    fi
    # get the difference between the current time and the last time
    # $(()) for bash arithmetic
    difference="$((linearray[0] - last))"
    # if the difference is smaller than the allowed difference, then run some command
    if ((difference < allow_difference)); then
        echo "difference between ${last} and ${linearray[0]} is ${difference}, running command"
        # could add some other command here...
    fi
    last="${linearray[0]}"
done

Tried to comment more than usual as you said you're new to bash but let me know if something isn't understandable.

 $ ./script 
difference between 20230913200618 and 20230913200620 is 2, running command
^C