bashawk

While using AWK, I'd like to verify that the NF is >= 10. If not, write the line to a reject file


I have an AWK script that is embedded in BASH code.

cat MyFile.csv | awk -v reject_file=MyRejectFile.csv "${AWK_SCRIPT}" | <generic loader>

And here is my AWK_SCRIPT...

   AWK_SCRIPT='BEGIN {
   FS=","
   OFS=","
   }
   {
      if (NF < 10) {
         print $0 >> reject_file
      }
   print $0
   }
   '

MyFile.csv looks something like this...

This,is,a,sample,line,that,has,more,than,10,fields
This,line,has,less,than,10,fields
This,is,a,second,line,that,has,more,than,10,fields
This,line,also,has,less,than,10,fields

I am trying to get the first and third lines to print to stdout and the second and fourth lines to print to the reject file.


Solution

  • There's a few ways to skin this cat:

    You could use an else:

       AWK_SCRIPT='BEGIN {
       FS=","
       OFS=","
       }
       {
          if (NF < 10) {
             print $0 >> reject_file
          }
          else {
             print $0
          }
       }
       '
    

    You could also just drop next inside the if block to stop processing the current line and skip to the next line:

       AWK_SCRIPT='BEGIN {
       FS=","
       OFS=","
       }
       {
          if (NF < 10) {
             print $0 >> reject_file
             next
          }
       print $0
       }
       '
    

    You could also just spell out each condition and resulting action:

       AWK_SCRIPT='BEGIN {
       FS=","
       OFS=","
       }
       NF < 10 {
          print $0 >> reject_file
       }
       NF >= 10 {
          print $0
       }
       '