unixpipecsplit

pipe output of gzip into csplit

This command works:

csplit really_big_file.txt -f ../dump/really_big_file_ /^H\|756\|/ {*}

but this command:

gzip -dc  really_big_file.txt.gz | csplit -f ../dump/really_big_file_ /^H\|756\|/ {*}

yields:

csplit: cannot open '/^H|756|/' for reading: No such file or directory

How to pipe output of gzip into csplit?

Solution

  • In man csplit it says that file always comes before pattern:

    SYNOPSIS

       csplit [OPTION]... FILE PATTERN...

    So it should be:

    gzip -dc  really_big_file.txt.gz | csplit -f ../dump/really_big_file_ - /^H\|756\|/ {*}

    Example:

    $ gzip -dc inputfile.txt.gz
abc
searchstring
def
searchstring
egh
searchstring
$ mkdir split
$ gzip -dc inputfile.txt.gz | csplit  -f split/file - /searchstring/ {*}
$ ls -Al split/
total 16
-rw-r--r-- 1 ja users  4 Sep 21 17:53 file00
-rw-r--r-- 1 ja users 17 Sep 21 17:53 file01
-rw-r--r-- 1 ja users 17 Sep 21 17:53 file02
-rw-r--r-- 1 ja users 13 Sep 21 17:53 file03
$ cat split/*
abc
searchstring
def
searchstring
egh
searchstring