javascriptnode.jsjsonsettimeoutpromise.all

How to execute the writeFileSync() only after the Promise.all() loop completion along with setTimeOut() using javascript?


I am reading a .txt file data and calling the 3rd party api to fetch more data using .txt file data as arguments continuously. For that I am running an await Promise.all() with map() loop using setTimeOut() function of delay 2 seconds so that 3rd party API gets latency time and avoid catch error.

After that I am appending/pushing it to a json object array. After that Writing the whole JSON.stringify(data) to a .json file. I want everything in a sequence. But unfortunately while debugging, what I see is that the writeFileSync gets executed even before loop completion which I dont want.

Here is my code I am trying:

const writeFile = async (obj) => {
  const json = JSON.stringify(obj);
  fs.writeFileSync('/home/deb/Downloads/Twitty-Bird/src/utils/output.json', json, 'utf8')
  return 'completed';
}

export const convertToJSONFile = async () => {
  try {
    let obj = {
      table: []
    };
    const data = fs.readFileSync('/home/deb/Downloads/Twitty-Bird/src/utils/sample.txt', 'utf8');
    if (!data) throw err;
    let splitted = data.toString().split("\n");
    let interval = 2000;
    await Promise.all(splitted.map(async (word, index) => {
      setTimeout(async function () {
        let wordMeaningDetails = await axios({
        method: 'GET',
        url: `https://api.dictionaryapi.dev/api/v2/entries/en/${word}`
        })
        wordMeaningDetails = wordMeaningDetails.data[0].meanings[0].definitions[0]
        obj.table.push({
          word: word, definition: wordMeaningDetails.definition, example: wordMeaningDetails.example
        });
      }, interval);
    }))

    const res = await writeFile(obj);
    console.log(res);
  }
  catch (err) {
    console.log("Error = ", err);
    //convertToJSONFile();
  }
}

convertToJSONFile();

What I want exactly in order in laymein terms:

  1. First read all data and split into array using fs.readFileSync
  2. Execute 3rd party api with axios one by one and append all data to object obj = {}
  3. Then finally write that json data to a .json file and save it in root folder.

Update: I am using this updated code now:

const promiseResponse = await Promise.all(splitted.map(async (word, index) => new Promise((resolve) => {
  setTimeout(async function () {
    let wordMeaningDetails = await findMeaning(word);
    wordMeaningDetails = wordMeaningDetails.data[0].meanings[0].definitions[0]
    obj.table.push({
      word: word, definition: wordMeaningDetails.definition, example: wordMeaningDetails.example
    });
    console.log(word);
    resolve(); // resolve the promise to mark it as "done"
  }, 1000 * index)
})
))

const res = await writeFile(obj);
console.log(res);

So after execution of whole splitted array and resolving the promises it throws the below error instead of executing the res = await writeFile(obj). I don't know why its happening.

aa
aardvark
aargh
aback
abacus
abandon
abandoned
abandoning
abandonment
abandons
(node:78808) UnhandledPromiseRejectionWarning: Error: Request failed with status code 404
    at createError (/home/vikas/Downloads/Twitty Bird/node_modules/axios/lib/core/createError.js:16:15)
    at settle (/home/vikas/Downloads/Twitty Bird/node_modules/axios/lib/core/settle.js:17:12)
    at IncomingMessage.handleStreamEnd (/home/vikas/Downloads/Twitty Bird/node_modules/axios/lib/adapters/http.js:293:11)
    at IncomingMessage.emit (events.js:412:35)
    at endReadableNT (internal/streams/readable.js:1334:12)
    at processTicksAndRejections (internal/process/task_queues.js:82:21)
(node:78808) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:78808) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Solution

  • You need to return a Promise in the map function. See below:

    await Promise.all(splitted.map(async (word, index) => ...)));
    // You need to return a promise not a anonymous function because the function
    // will resolve instantely and is not waiting for your timeout
    

    (async () => {
    
      await Promise.all([1, 2, 3].map((word, index) => new Promise((resolve) => {
        setTimeout(async function() {
          console.log(word);
          // do your api stuff
          resolve(); // resolve the promise to mark it as "done"
        }, 1000 * index)
      })))
      console.log("done!")
    
    })();