node.jsasynchronousparallel-processing

how to use async.parallelLimit to maximize the amount of (parallel) running processes?


Is it possible to set a Limit to parallel running processes with async.parallelLimit? I used following very simple code to test how it works.

var async = require("async");
var i = 0;

function write() {
    i++;
    console.log("Done", i);
}

async.parallelLimit([
    function(callback) {
        write();
        callback();
    }
], 10, function() {
    console.log("finish");
});

Of course everything what I got back was this:

Done 1
finish

In my case I have a function which is called very often, but I only want it to run only 5 times simultaneously. The other calls should be queued. (Yes I know about async.queue but, this is not what I want in this question).

So now the question is, is this possible with async.parallelLimit?

//EDIT:

I actually have something similar to this:

for(var i = 0; i < somthing.length; i++) { //i > 100  for example
    write();
}

And because of the non synchronicity of node.js this could run 100 times at the same time. But how shall I limit the parallel running processes in this case?


Solution

  • Very short answer; Yes. That's exactly what asyncParallelLimit does.

    In your case, you are passing only one function to parallelLimit. That's why it only get's called once. If you were to pass an array with this same function many times, it will get executed as many times as you put it in the array.

    Please note that your example function doesn't actually do any work asynchronously. As such, this example function will always get executed in series. If you have a function that does async work, for example a network request or file i/o, it will get executed in parallel.

    A better example-function for a async workload would be:

    function(callback) {
        setTimeout(function(){
            callback();
        }, 200);
    }