bashunixscripting

Delete all but the most recent X files in bash


Is there a simple way, in a pretty standard UNIX environment with bash, to run a command to delete all but the most recent X files from a directory?

To give a bit more of a concrete example, imagine some cron job writing out a file (say, a log file or a tar-ed up backup) to a directory every hour. I'd like a way to have another cron job running which would remove the oldest files in that directory until there are less than, say, 5.

And just to be clear, there's only one file present, it should never be deleted.


Solution

  • The problems with the existing answers:

    wnoise's answer addresses these issues, but the solution is GNU-specific (and quite complex).

    Here's a pragmatic, POSIX-compliant solution that comes with only one caveat: it cannot handle filenames with embedded newlines - but I don't consider that a real-world concern for most people.

    For the record, here's the explanation for why it's generally not a good idea to parse ls output: http://mywiki.wooledge.org/ParsingLs

    ls -tp | grep -v '/$' | tail -n +6 | xargs -I {} rm -- {}
    

    Note: This command operates in the current directory; to target a directory explicitly, use a subshell ((...)) with cd:
    (cd /path/to && ls -tp | grep -v '/$' | tail -n +6 | xargs -I {} rm -- {})
    The same applies analogously to the commands below.

    The above is inefficient, because xargs has to invoke rm separately for each filename.
    However, your platform's specific xargs implementation may allow you to solve this problem:


    A solution that works with GNU xargs is to use -d '\n', which makes xargs consider each input line a separate argument, yet passes as many arguments as will fit on a command line at once:

    ls -tp | grep -v '/$' | tail -n +6 | xargs -d '\n' -r rm --
    

    Note: Option -r (--no-run-if-empty) ensures that rm is not invoked if there's no input.

    A solution that works with both GNU xargs and BSD xargs (including on macOS) - though technically still not POSIX-compliant - is to use -0 to handle NUL-separated input, after first translating newlines to NUL (0x0) chars., which also passes (typically) all filenames at once:

    ls -tp | grep -v '/$' | tail -n +6 | tr '\n' '\0' | xargs -0 rm --
    

    Explanation:


    A variation on the original problem, in case the matching files need to be processed individually or collected in a shell array:

    # One by one, in a shell loop (POSIX-compliant):
    ls -tp | grep -v '/$' | tail -n +6 | while IFS= read -r f; do echo "$f"; done
    
    # One by one, but using a Bash process substitution (<(...), 
    # so that the variables inside the `while` loop remain in scope:
    while IFS= read -r f; do echo "$f"; done < <(ls -tp | grep -v '/$' | tail -n +6)
    
    # Collecting the matches in a Bash *array*:
    IFS=$'\n' read -d '' -ra files  < <(ls -tp | grep -v '/$' | tail -n +6)
    printf '%s\n' "${files[@]}" # print array elements