javascriptnode.jsfunctionpromisedestroy

Node JS: How to properly end/destroy a stream within a promise of a function


I would like to know why on .destroy() will no longer free my memory if a promise is within a function.

Secondly, I would like to know either a proper way to on .destroy a promise within a function or pass values to promise without requiring a function.

It's easy to get a promise to end/destroy if it is not in a function- But I need to pass info to the promise object and don't know any other way of doing that without wrapping a function around it. The problem is once the function is wrapped around the promise, the end/destroy call of the promise is no longer detected.


THIS WORKS: I can correctly end a stream within a promise with the code below:

const p1= new Promise((resolve, reject) => {
 
  let readStream = readline.createInterface({
    input: fs.createReadStream('pathtofile.txt','utf8')
  });

  readStream.on("line", (line) => {
    //READ LARGE FILE HERE, LINE BY LINE
  });
    
  readStream.on('end', () => {
    readStream.destroy(); /*frees memory*/
  });

  readStream.on("close", () =>
    resolve({
      RETURNVALUE
    }) 
  )
});

Promise.all([p1]).then((results) => {console.log(results)};

THIS DOESN'T WORK: If I wrap a function around promise to pass values, .on end/destroy no longer works (thus heap errors are thrown):

const p1 = function(value1,value2,value3){
   return new Promise((resolve, reject) => {
     let readStream = readline.createInterface({
       input: fs.createReadStream('pathtofile.txt','utf8')
     });
    
    readStream.on("line", (line) => {
      //READ LARGE FILE HERE, LINE BY LINE
    });
        
    readStream.on('end', () => {
      readStream.destroy();   /*No longer frees memory*/
    });
    
    readStream.on("close", () =>
      resolve({
        RETURNVALUE
      }) 
    )
  });
}
    
Promise.all([p1(v1,v2,v3]).then((results) => {console.log(results)};

Solution

  • Turns out the coding above was entirely correct and I was wrong as to the reason why my script was ending up with EMFILE errors such as "too many files are open". The issue was that I am on MAC OS Monterey with a default of 256 limit of file descriptors that can be opened at one time. I upped my ulimit -n from 256 to 150000 and everything worked fine.

    I used the resource below to up the limit: Super User: How do I increase the max open files in macOS Big Sur?