linuxshellshlsof

How to identify if file is still written or completed through linux script


We have one system which is generating files and I want to check which files out of many files are completed and also not been modified from past two minutes through a script and finally rename those.

This is what I tried but the result is not correct. Could someone help?

for file in /home/test/*abc_YYYYMMDDhhmmss*
do
    f1=`basename $file`
    if [ lsof | grep "$f1" = "" ];then
        if  [ `stat --format=%Y $file` -le $(( `date +%s` - 300 )) ]; then
        mv "$f1" "${f1}_Complete"
    else
       echo "no files to collect"
    fi
done

Solution

  • You are making the common mistake of assuming that [ is part of the if command's syntax; but it's not: [ is just another command. The syntax for an if statement is

    if commands; then
        : what to do if the exit code from commands was 0
    else
        : what to do if not
    fi
    

    where commands can be an arbitrarily complex sequence of commands, and the exit code from the last command in the sequence decides which branch to take; and the else branch is optional.

    As a minimal fix, change to

        # use modern $(command substitution) syntax
        # instead of obsolescent `command substitution`;
        # always quote variables with file names
        f1=$(basename "$file")
        # Remove [ and switch to grep -q;
        # add -F to grep flags for literal matching
        if ! lsof | grep -Fq "$f1"; then
    

    Anyway, what about something like this instead?

    find $(lsof |
        awk 'NR==FNR { if ($9 ~ /^\/home\/test\//) a[$9]++; next }
        FNR == 1 {
            if (! (FILENAME in a)) print FILENAME;
            next }' - /home/test/*abc_YYYYMMDDhhmmss*) \
        -type f -mmin +2 -exec sh -c '
            for file; do
                mv "$file" "${file}_Complete"
            done' _ {} +
    

    This is pretty complex, but here's a rundown.

    This will probably not work if your file names contain newlines; then you'll need something more complex still.

    Looping over arbitrary file names is disappointingly complex in Bourne-family shells, and finding elements not in a list is always slightly pesky in shell script. Ksh and Bash offer some relief because they have arrays, but this is not portable to POSIX sh / ash / dash.