phpcforkprocess-management

How can I enforce a maximum amount of forked children?

EDIT: I've tagged this C in a hope to get more response. It's more the theory I'm interested in than a specific language implementation. So if you're a C coder please treat the following PHP as pseudo-code and feel free to respond with an answer written in C.

I am trying to speed up a PHP CLI script by having it execute its tasks in parallel instead of serial. The tasks are completely independent of each other so it doesn't matter which order they start/finish in.

Here's the original script (note all these examples are stripped-back for clarity):

<?php

$items = range(0, 100);

function do_stuff_with($item) { echo "$item\n"; }

foreach ($items as $item) {
    do_stuff_with($item);
}

I've managed to make it work on the $items in parallel with pcntl_fork() as shown below:

<?php

ini_set('max_execution_time', 0); 
ini_set('max_input_time', 0); 
set_time_limit(0);

$items = range(0, 100);

function do_stuff_with($item) { echo "$item\n"; }

$pids = array();
foreach ($items as $item) {
    $pid = pcntl_fork();
    if ($pid == -1) {
        die("couldn't fork()");
    } elseif ($pid > 0) {
        // parent
        $pids[] = $pid;
    } else {
        // child
        do_stuff_with($item);
        exit(0);
    }   
}

foreach ($pids as $pid) {
    pcntl_waitpid($pid, $status);
}

Now I want to extend this so there's a maximum of, say, 10 children active at once. What's the best way of handling this? I've tried a few things but haven't had much luck.

Solution

  • There is no syscall to get a list of child pids, but ps can do it for you.

    --ppid switch will list all children for you process so you just need to count number of lines outputted by ps.

    Alternatively you can maintain your own counter that you will increment on fork() and decrement on SIGCHLD signal, assuming ppid stays unchanged for fork'ed processed.