I would like to know why on .destroy() will no longer free my memory if a promise is within a function.
Secondly, I would like to know either a proper way to on .destroy a promise within a function or pass values to promise without requiring a function.
It's easy to get a promise to end/destroy if it is not in a function- But I need to pass info to the promise object and don't know any other way of doing that without wrapping a function around it. The problem is once the function is wrapped around the promise, the end/destroy call of the promise is no longer detected.
THIS WORKS: I can correctly end a stream within a promise with the code below:
const p1= new Promise((resolve, reject) => {
let readStream = readline.createInterface({
input: fs.createReadStream('pathtofile.txt','utf8')
});
readStream.on("line", (line) => {
//READ LARGE FILE HERE, LINE BY LINE
});
readStream.on('end', () => {
readStream.destroy(); /*frees memory*/
});
readStream.on("close", () =>
resolve({
RETURNVALUE
})
)
});
Promise.all([p1]).then((results) => {console.log(results)};
THIS DOESN'T WORK: If I wrap a function around promise to pass values, .on end/destroy no longer works (thus heap errors are thrown):
const p1 = function(value1,value2,value3){
return new Promise((resolve, reject) => {
let readStream = readline.createInterface({
input: fs.createReadStream('pathtofile.txt','utf8')
});
readStream.on("line", (line) => {
//READ LARGE FILE HERE, LINE BY LINE
});
readStream.on('end', () => {
readStream.destroy(); /*No longer frees memory*/
});
readStream.on("close", () =>
resolve({
RETURNVALUE
})
)
});
}
Promise.all([p1(v1,v2,v3]).then((results) => {console.log(results)};
Turns out the coding above was entirely correct and I was wrong as to the reason why my script was ending up with EMFILE errors such as "too many files are open". The issue was that I am on MAC OS Monterey with a default of 256 limit of file descriptors that can be opened at one time. I upped my ulimit -n from 256 to 150000 and everything worked fine.
I used the resource below to up the limit: Super User: How do I increase the max open files in macOS Big Sur?