[SOLVED] Timestamp and data on different lines

Timestamp and data on different lines

My datafile is in the following format:

2024-11-11 11:08:39
[1]123,45,67,89,0,0,0
[2]11,22,33,44,0,0,0
[5]99,88,77,66,0,0,0

2024-11-11 11:08:40
[1]123,45,67,89,0,0,0
[2]55,66,77,88,0,0,0
[5]99,88,77,66,0,0,0

The data file is appended once per second with a similar block of data. I want to make a rolling plot as in Dynamic Gnuplot graphs of a parameter, for example, line [2] parameter 3, using the first line timestamp of the block as x-axis value.

How to use a single x-axis value for multiple following lines y-axis values?

Solution

Although a few details are missing in your question, here is a suggestion as a starting point. Your data format is a bit "special", but it can be handled. Small changes to your format might simplify the script, however, let's start with the data format as provided.

The basic principle of the rolling plot is taken from here. Only your data is a bit different.

Just a few comments without going too much into details:

to my knowledge, gnuplot has no intrinsic way to plot the N last data points. Hence, you have to either use an external tool or to use the gnuplot feature stats which does some statistics on the file. With this, you can find out the number of lines, blocks, subblocks, and furthermore minimum, maximum, mean, etc. (check help stats). If your file is huge (Mega or GigaBytes) this might take some time.
the pseudocolumn -1, i.e. column(-1), (check help pseudocolumns) holds the index value b1 (zero-based) of the subblock. So, if this number changes, i.e. b0!=b1, you have the first line of a subblock which contains the date/time. Store this value in t0.
you can set datafile separator "[]," which will split the data into columns at the given characters. Hence your selector in [] will be in column 2 and your "parameter n" will be in column n+2.
if you need the data points connected via line, there will be a workaround necessary (not implemented here), because gnuplot interrupts lines when there is an empty line in the data (which is the case here).

Since I don't have an external source creating some data, some test data is created within the script itself. If you have your file updated externally by another program skip the lines marked with # *skip.

The data in the file SO79180869.dat looks something like this:

2024-11-12 16:26:14
[1]73,8,16
[2]6,20,29
[5]23,17,39

2024-11-12 16:26:15
[1]45,92,68
[2]72,7,47
[5]8,36,48

2024-11-12 16:26:16
[1]71,35,66
[2]37,9,34
[5]17,62,93

...

Script:

### dynamic plotting of last N points of special data format
reset session

FILE = "SO79180869.dat"

set print FILE          # *skip these lines if your file is updated by another program
set print FILE append   # *skip

N = 10                  # plot last N values
bind x 'stop = 1'       # stop loop by pressing "x"
stop = 0
set yrange [0:100]

set datafile separator "[],"
myFilter(q,p) = column(2)==q ? column(p+2) : NaN
myFmt         = "%Y-%m-%d %H:%M:%S"
myDate(col)   = (b0=b1, b1=column(-1), b0==b1 ? t0 : t0=timecolumn(col,myFmt))

set format x "%H:%M:%S" timedate
set xrange [:] noextend

while (!stop) {

    print strftime(myFmt, time(0))                                     # *skip
    print sprintf("[1]%d,%d,%d",rand(0)*100,rand(0)*100,rand(0)*100)   # *skip
    print sprintf("[2]%d,%d,%d",rand(0)*100,rand(0)*100,rand(0)*100)   # *skip
    print sprintf("[5]%d,%d,%d",rand(0)*100,rand(0)*100,rand(0)*100)   # *skip
    print ""                                                           # *skip

    pause 1.0                                             # pause in seconds
    stats [*:*][*:*] FILE u (N0=column(-1)+1) nooutput    # analyze file and get number of subblocks
    N1 = N0<N ? 0 : N0-N                                  # line index of N-last line

    plot b1=t0=NaN FILE u (myDate(1)):(myFilter(5,3)) every :::N1 \
        w lp pt 7 ti sprintf("Last %d of %d values",N0-N1,N0)
}
set print               # *skip
### end of script

Result: (screen capture with ScreenToGif from wxt terminal, 2x faster)