linuxbashlarge-filesfilesplitting

Splitting a file and its lines under Linux/bash

I have a rather large file (150 million lines of 10 chars). I need to split it in 150 files of 2 million lines, with each output line being alternatively the first 5 characters or the last 5 characters of the source line. I could do this in Perl rather quickly, but I was wondering if there was an easy solution using bash. Any ideas?

Solution

  • Homework? :-)

    I would think that a simple pipe with sed (to split each line into two) and split (to split things up into multiple files) would be enough.

    The man command is your friend.

    Added after confirmation that it is not homework:

    How about

    sed 's/\(.....\)\(.....\)/\1\n\2/' input_file | split -l 2000000 - out-prefix-

    ?