linuxsedcopyverticavsql

Remove footer from Vertica's Copy Script

I have a vertica copy script (A.copy.vsql) which is loading data into a table from a file which has HEADER, TRAILER and DETAIL RECORDS.

Vertica Copy statement can skip 1 record which means I know how to remove the header.

I want to know if I can chop the trailer in the same way or not?

Also, if I cannot chop it like the header then can I write simple linux SED commands in the VSQL itself (A.copy.vsql) to do the job?

Solution

  • If you want to remove the first and last line from a file you can use:

    sed '1d;$d' file

    And used in a command, with the pattern command file, one can, with bash, use a process substitution:

    command <(sed '1d;$d' file)

    1 and $ are absolute addresses, 1 means first line, while $ means last. d deletes the line addressed.

    <(...) is a process substitution.