shellawkpipetail

Auto-insert blank lines in `tail -f`


Having a log file such as:

[DEBUG][2016-06-24 11:10:10,064][DataSourceImpl] - [line A...]
[DEBUG][2016-06-24 11:10:10,069][DataSourceImpl] - [line B...]
[DEBUG][2016-06-24 11:10:12,112][DataSourceImpl] - [line C...]

which is under tail -f real-time monitoring, is it possible to auto-insert (via a command we would pipe to the tail) "blank lines" after, let's say, 2 seconds of inactivity?

Expected result:

[DEBUG][2016-06-24 11:10:10,064][DataSourceImpl] - [line A...]
[DEBUG][2016-06-24 11:10:10,069][DataSourceImpl] - [line B...]
---
[DEBUG][2016-06-24 11:10:12,112][DataSourceImpl] - [line C...]

(because there is a gap of more than 2 seconds between 2 successive lines).


Solution

  • awk -F'[][\\- ,:]+' '1'
    

    The above will split fields on ], [, -, ,, and :, so that each field is as described below:

    [DEBUG][2016-06-24 11:10:10,064][DataSourceImpl] - [line A...]
     22222  3333 44 55 66 77 88 999  ...
    

    You can then concatenate some of the fields and use that to measure time difference:

    tail -f input.log | awk -F'[][\\- ,:]+' '{ curr=$3$4$5$6$7$8$9 }
                          prev + 2000 < curr { print "" } # Print empty line if two seconds 
                                                          # have passed since last record.
                                             { prev=curr } 1'