bashcachingcommand-linecommand-line-interface

Can i cache the output of a command on Linux from CLI?


I'm looking for an implementation of a 'cacheme' command, which 'memoizes' the output (stdout) of whatever has in ARGV. If it never ran it, it will run it and somewhat memorize the output. If it ran it, it will just copy the output of the file (or even better, both output and error to &1 and &2 respectively).

Let's suppose someone wrote this command, it would work like this.

$ time cacheme sleep 1    # first time it takes one sec
real   0m1.228s
user   0m0.140s
sys    0m0.040s

$ time cacheme sleep 1    # second time it looks for stdout in the cache (dflt expires in 1h)
#DEBUG# Cache version found! (1 minute old)

real   0m0.100s
user   0m0.100s
sys    0m0.040s

This example is a bit silly because it has no output. Ideally it would be tested on a script like sleep-1-and-echo-hello-world.sh.

I created a small script that creates a file in /tmp/ with hash of full command name and username, but I'm pretty sure something already exists.

Are you aware of any of this?

Note. Why I would do this? Occasionally I run commands that are network or compute intensive, they take minutes to run and the output doesn't change much. If I know it in advance I'd just prepend a cacheme <cmd>, go for dinner and when i'm back I can just rerun the SAME command over and over on the same machine and get the same answer (meaning, same stdout) in an instance.


Solution

  • Improved solution above somewhat by also adding expiry age as optional argument.

    #!/bin/sh
    # save as e.g. $HOME/.local/bin/cacheme
    # and then chmod u+x $HOME/.local/bin/cacheme
    VERBOSE=false
    PROG="$(basename $0)"
    DIR="${HOME}/.cache/${PROG}"
    mkdir -p "${DIR}"
    EXPIRY=600 # default to 10 minutes
    # check if first argument is a number, if so use it as expiration (seconds)
    [ "$1" -eq "$1" ] 2>/dev/null && EXPIRY=$1 && shift
    [ "$VERBOSE" = true ] && echo "Using expiration $EXPIRY seconds"
    CMD="$@"
    HASH=$(echo "$CMD" | md5sum | awk '{print $1}')
    CACHE="$DIR/$HASH"
    test -f "${CACHE}" && [ $(expr $(date +%s) - $(date -r "$CACHE" +%s)) -le $EXPIRY ] || eval "$CMD" > "${CACHE}"
    cat "${CACHE}"