I have an express application where I have a generator function that needs aprox. 5 minutes for processing a lot of data. Unfortunately I can not optimize that function.
Express automatically times out after 2 minutes and I do not want to alter that only for this specific function. I figured that maybe if I made periodically a res.write() call, the 2 minute rule does not apply.
My question:
How can I execute a res.write('Something') every X seconds while waiting for the other function to terminate ?
I want it to do something like the following, I hope you get the idea.
function main() {
co(function* () {
const something = yield promise(); // function that needs a lot of time
const doWhileWaiting = setTimeout(() => {
if (!something) {
// Print this while waiting for the value of something
console.log('Waiting for something ... ');
} else {
console.log(something);
clearInterval(doWhileWaiting);
}
}, 5000);
});
}
You should not have such a heavy, blocking function in your api itself. Much better would be to have another process handle this sort of computing for you as node is single threaded and this will block all other computation. There sure are ways to do this in one thread/process by adding timeout in your computing functions, but that's not very nice style. As i don't know what your function is doing, it's hard to give a solid answer what's best in your case.
As you say its executed once a day, you could do a cronjob trigger another process. If you need to communicate to your api process, you might want to look at child processes. https://nodejs.org/api/child_process.html