javascriptcsvdictionaryparsingpapaparse

Map object not setting values after execution of function


I have a function below where the purpose is to parse multiple local csv files. For each parsed csv file, I then grab 1-5 random elements. Since the CSV parser outputs the data in a nested array structure ([['a'], ['b'], ['c']]), I further process the data and extract the individual array elements to push into the final array.

My final goal is to update the values in the types map by passing in the respective arrays for each of its keys

let health = createHealthData();
console.log(health, 'final output')

function createHealthData() {
  let types = new Map();
  types.set('allergies', '');
  types.set('medications', '')
  let start = 1;
  let limit = 5;

  for (let key of types.keys()) {
    let randNum = Math.floor(Math.random() * (limit - start + 1) + start); // Generates random numbers between start and limit
    let file = fs.createReadStream(path.resolve(__dirname, `${key}.csv`))

    Papa.parse(file, {
      header: false,
      complete: (results) => {
        let first = []
        let data = results.data;

        let final = [];
        for (let i = 1; i <= randNum; i++) {
          first.push(data[Math.floor(Math.random() * data.length)])
        }

        first.forEach((x) => {
          x.forEach((y) => {
            final.push(y)
          })
        })
        types.set(key, final);
        console.log(types);
      }
    })
  }
  return types;
}

My function parses and processes the csv files fine, but it refuses to update the existing keys in the map once I return the types map

For instance, my output at console.log(types) gives me:

Map(2) {
   allergies => ['Banana Allergy'],
   medications => ['Opana ER', 'Atrovent HFA']
}

which is what I intend to pass out of the function.

However, upon viewing the health variable after the function finishes executing, the output is:

Map(2) {'allergies' => '', 'medications' => ''} final output

as if the line types.set(key,final) never executed at the end of the function. I'm struggling to find what the source of this bug is and based on the scope of the types variable in the function, I'm not seeing where it might be overwritten.


Solution

  • It looks like Papa.parse is asynchronous: https://www.papaparse.com/docs

    "Doesn't return anything. Results are provided asynchronously to a callback function."

    Since you return types at the end, you won't get the proper values yet due to the asynchronous nature of the execution (i.e. return types happens first, followed by the parsing complete callback).

    There's a similar question that describes different ways of handling Papa.parse: How to use Promises with PapaParse?