[SOLVED] Bash/GNU Tar - pipe output to function

Bash/GNU Tar - pipe output to function

I'm trying to extract a common tar.gz file. I currently have:

function streamLog()
{
    if [ -n "$1" ]
    then
        IN="$1"
    else
        read IN # This reads a string from stdin and stores it in a variable called IN
    fi

    if [ -z "${IN}" ]
    then
        return 0
    else
        echo "${IN}" >> zlib.log
    fi
}

/bin/tar --no-same-owner -zxvf ~/src/zlib-1.2.11.tar.gz -C /build | streamLog

This causes a lot of files contained within zlib to fail to extract. Weirdly some do though. I see no output send to my streamLog function which are unexpected. This so far is the only command I've had fail using this method.

I expect tar might be a special case. Why would this happen?

What should I do to correctly get the output of a command streamed to a function?

Solution

This works for me:

function streamLog()
{
  IFS=
  while read -r IN
  do
    echo "$IN" >> zlib.log
  done
}

/bin/tar --no-same-owner -zxvf 1.tar.gz -C build | streamLog

You say you see no output sent to your zlib.log file. The original code read from standard input correctly but only for one line. tar -vf will likely produce multiple lines of output as a tar file will probably have multiple files within it.

If files are failing to extract, do you have the permission to save that file into the filesystem at the fullpath name of the file, watching out for paths that start with '/'? To test permissions, can you run using sudo?

What should I do to correctly get the output of a command streamed to a function?
The output of tar is put onto standard output which is piped into the function which sits in a while loop, reading from standard input until that stream is closed (in our case, the standard input stream will be closed when the -vf information from tar is complete).

Every line of standard input ends up in the IN variable that is then echoed and appended to zlib.log.

IFS is set to nothing so that read does not split the input (output from tar) into words.
read -r is used so that read does not interpret any backslashes as escape characters.