javascriptnode-csv-parse

csv-parse is throwing Invalid Opening Quote: a quote is found inside a field at line


I know there are other posts out there but none of them seem to fix my issues. I am using csv-parse with node js. This is the CSV header and record that I'm trying to parse.

sku,description,productUnitOfMeasure,unitCost,retailPrice,wholesalePrice,dropShipPrice,assemblyCost,planner,comments,productId,fileUpdate,SkuStatus,Master Planning Family,Category,Sub-Category,Brand,ShortCode,Import/Domestic,Inventory Value,Master Pack Quantity,Pallet / TI / HI,40HC Quantity,Product Group ID
032406021945-GreenBay,TFAL B2080264 INIT TNS GRY SAUTE PN 8",EA,7.72,13.99,0.00,0.00,0,Whitney Ehlke-2307,,032406021945,2022-01-25,New,COOKWARE,OPENSTOCK,NONE,T-FAL,B2080264,Domestic,208.44,3,0/0/0,0,23

I have no control over this file. I just need to be able to parse it. You will see that there is a double quote at the end of the description: TFal B2080264 INI TNS GRY SAUTE PN 8".

I need the double quote to stay there and for that to parse as one field. I keep getting this error:

Invalid Opening Quote: a quote is found inside a field at line 2.

The quote is not an opening. It's technically a closing. But regardless, it will not parse.

This is currently my code:

const parser = fs.createReadStream(filePath).pipe(
    parse({ columns: true, relax_quotes: true, escape: '\\', ltrim: true, rtrim: true })
)

I have removed some of the params and tried others, to no avail. Any ideas??


Solution

  • This code works fine with the latest csv-parse version (5.0.4). Which version of the csv-parse package are you using? I ask because it looks like the option may have been renamed from relax to relax_quotes only recently.

    So, I think the solution is either:

    1. upgrade to the latest csv-parse, and indicate relax_quotes, or
    2. stay with your current version of csv-parse, and indicate relax

    Just to be sure relax_quotes works with the current library, I tested the following code and it worked as expected:

    const csv = require('csv-parse');
    const fs = require('fs');
    
    const parser = fs.createReadStream("70880341.csv").pipe(
      csv.parse({ columns: true, relax_quotes: true, escape: '\\', ltrim: true, rtrim: true })
    )
    
    const records = [];
    
    parser.on('readable', function() {
      let record;
      while ((record = parser.read()) !== null) {
        records.push(record);
      }
    });
    
    parser.on('error', function(err) {
      console.error(err.message);
    });
    
    parser.on('end', function() {
      console.log(records);
    });
    

    Result:

    [{
        sku: '032406021945-GreenBay',
        description: 'TFAL B2080264 INIT TNS GRY SAUTE PN 8"',
        productUnitOfMeasure: 'EA',
        ...
    }]