bashfile-processinggnu-parallel

Splitting command line args with GNU parallel

Using GNU parallel: http://www.gnu.org/software/parallel/

I have a program that takes two arguments, e.g.

$ ./prog file1 file2
$ ./prog file2 file3
...
$ ./prog file23456 file23457

I'm using a script that generates the file name pairs, however this poses a problem because the result of the script is a single string - not a pair. like:

$ ./prog "file1 file2"

GNU parallel seems to have a slew of tricks up its sleeves, I wonder if there's one for splitting text around separators:

$ generate_file_pairs | parallel ./prog ?  
  # where ? is text under consideration, like "file1 file2"

The easy work around is to split the args manually in prog, but I'd like to know if it's possible in GNU parallel.

Solution

  • You are probably looking for --colsep.

    generate_file_pairs | parallel --colsep ' ' ./prog {1} {2}

    Read man parallel for more. And watch the intro video if you have not already done so http://www.youtube.com/watch?v=OpaiGYxkSuQ