bashunixinotifyinotifywait

Monitor Pre-existing and new files in a directory with bash

I have a script using inotify-tool.
This script notifies when a new file arrives in a folder. It performs some work with the file, and when done it moves the file to another folder. (it looks something along these line):

inotifywait -m -e modify "${path}" |
    while read NEWFILE
       work on/with NEWFILE
       move NEWFILE no a new directory
    done

By using inotifywait, one can only monitor new files. A similar procedure using for OLDFILE in path instead of inotifywait will work for existing files: 

for OLDFILE in ${path} 
do 
   work on/with OLDFILE 
   move NEWFILE no a new directory
done

I tried combining the two loops. By first running the second loop. But if files arrive quickly and in large numbers there is a change that the files will arrive wile the second loop is running. These files will then not be captured by neither loop.

Given that files already exists in a folder, and that new files will arrive quickly inside the folder, how can one make sure that the script will catch all files?

Solution

  • Once inotifywait is up and waiting, it will print the message Watches established. to standard error. So you need to go through existing files after that point.

    So, one approach is to write something that will process standard error, and when it sees that message, lists all the existing files. You can wrap that functionality in a function for convenience:

    function list-existing-and-follow-modify() {
  local path="$1"
  inotifywait --monitor \
              --event modify \
              --format %f \
              -- \
              "$path" \
    2> >( while IFS= read -r line ; do
            printf '%s\n' "$line" >&2
            if [[ "$line" = 'Watches established.' ]] ; then
              for file in "$path"/* ; do
                if [[ -e "$file" ]] ; then
                  basename "$file"
                fi
              done
              break
            fi
          done
          cat >&2
        )
}

    and then write:

    list-existing-and-follow-modify "$path" \
| while IFS= read -r file
    # ... work on/with "$file"
    # move "$file" to a new directory
  done

    Notes: