I need to grab the exit status from a command that I run via xargs
. Since that's not possible it's recommended to use parallel
. Some documentation suggests that it's often as simple as swapping the usage of xargs
with parallel
. My usage of xargs
is so simple that I would think this applies to me.
To restate my post-title, the ONLY reason I want to use parallel
is to be able to parse the file created with --joblog
. In other words, I don't actually want to run anything in parallel.
I am failing to figure out how to do what I would think would be a simple task. Google-searches have struck out for me finding an answer. I'm hoping someone here can put me to shame and show me how simple this is.
Here's some boiled-down test code to demonstrate the issue I'm bumping up against.
#!/bin/bash
tmpDir=/tmp/tmpTest.$$
tmpArgs=/tmp/tmpArgs.$$
tmpJobLog=/tmp/tmpJobLog.$$
mkdir $tmpDir
cd $tmpDir
touch aaa bbb 'c c' 'd d'
cat << EOF > $tmpArgs
--create
--file
test.tar
aaa
bbb
c c
d d
EOF
# I'm only using 'tar' for this simple test, it doesn't need to be 'tar',
# but it demonstrates my problem easily when comparing xargs vs parallel usage.
#
tr '\n' '\0' < $tmpArgs | xargs -0 tar
tar -t -f test.tar
/bin/rm test.tar
tr '\n' '\0' < $tmpArgs | parallel --joblog $tmpJobLog -0 tar
cat $tmpJobLog
/bin/rm -r /tmp/tmpTest.$$
/bin/rm $tmpArgs
/bin/rm $tmpJobLog
The xargs
line works as I'd expect, and the tar -t -f test.tar
prints out what you'd expect. That is:
aaa
bbb
c c
d d
However my simple minded substitution of parallel
for xargs
seems to want to run all the arguments to tar
in parallel! Creating a host of errors (of course):
tar: Cowardly refusing to create an empty archive
Try 'tar --help' or 'tar --usage' for more information.
tar: option '--file' requires an argument
Try 'tar --help' or 'tar --usage' for more information.
tar: invalid option -- 'e'
<SNIP>
tar: invalid option -- ' '
Try 'tar --help' or 'tar --usage' for more information.
Examining the joblog
file it's clear what's going on:
Seq Host Starttime JobRuntime Send Receive Exitval Signal Command
1 : 1746839006.000 0.000 0 0 2 0 tar --create
2 : 1746839006.000 0.000 0 0 64 0 tar --file
3 : 1746839006.000 0.000 0 0 64 0 tar test.tar
4 : 1746839006.000 0.000 0 0 2 0 tar aaa
5 : 1746839006.000 0.000 0 0 2 0 tar bbb
6 : 1746839006.000 0.000 0 0 64 0 tar 'c c'
7 : 1746839006.000 0.000 0 0 64 0 tar 'd d'
So that was a long winded way of asking, how do I simply get parallel
to behave like xargs
?
Thanks kindly!
As you don't show the arguments being chunked/grouped in any way, it seems you wish to have xargs
run a single command that consumes all the arguments.
If that is the case, simply running the command without wrapping it with xargs
will provide direct access to its exit code. For example:
mapfile -t <"$tmpArgs"
tar "${MAPFILE[@]}"
echo "exit code = $?"
Or even:
( set -f; IFS=$'\n'; tar $(<"$tmpArgs") )
Or for any POSIX shell:
( set -f; IFS='
'; tar $(cat "$tmpArgs") )
The absence of quoting around $(...)
activates word-splitting using IFS
(but set -f
prevents any pathname expansion). The sub-shell context makes the noglob and IFS
changes temporary, and sets $?
to the exit value of the final command within.
To use parallel
, the documentation says:
With --xargs GNU parallel will fit as many arguments as possible on a single line:
So:
tr '\n' '\0' "$tmpArgs" | parallel -0 --joblog "$tmpJobLog" --xargs tar