multithreadingperlforkmulticoremultiprocess

How can I make my Perl script use multiple cores for child processes?


I'm working on a mathematical model that uses data generated from XFOIL, a popular aerospace tool used to find the lift and drag coefficients on airfoils.

I have a Perl script that calls XFOIL repeatedly with different input parameters to generate the data I need. I need XFOIL to run 5,600 times, at around 100 seconds per run, soabout 6.5 days to complete.

I have a quad-core machine, but my experience as a programmer is limited, and I really only know how to use basic Perl.

I would like to run four instances of XFOIL at a time, all on their own core. Something like this:

while ( 1 ) {

    for ( i = 1..4 ) {

        if ( ! exists XFOIL_instance(i) ) {

            start_new_XFOIL_instance(i, input_parameter_list);
        }
    }
} 

So the program is checking (or preferably sleeping) until an XFOIL instance is free, when we can start a new instance with the new input parameter list.


Solution

  • Try Parallel::ForkManager. It's a module that provides a simple interface for forking off processes like this.

    Here's some example code:

    #!/usr/bin/perl
    
    use strict;
    use warnings;
    use Parallel::ForkManager;
    
    my @input_parameter_list = 
        map { join '_', ('param', $_) }
        ( 1 .. 15 );
    
    my $n_processes = 4;
    my $pm = Parallel::ForkManager->new( $n_processes );
    for my $i ( 1 .. $n_processes ) {
        $pm->start and next;
    
        my $count = 0;
        foreach my $param_set (@input_parameter_list) {         
            $count++;
            if ( ( $count % $i ) == 0 ) {
                if ( !output_exists($param_set) ) {
                    start_new_XFOIL_instance($param_set);
                }
            }
        }
    
        $pm->finish;
    }
    $pm->wait_all_children;
    
    sub output_exists {
        my $param_set = shift;
        return ( -f "$param_set.out" );
    }
    
    sub start_new_XFOIL_instance {
        my $param_set = shift;
        print "starting XFOIL instance with parameters $param_set!\n";
        sleep( 5 );
        touch( "$param_set.out" );
        print "finished run with parameters $param_set!\n";
    }
    
    sub touch {
        my $fn = shift;
        open FILE, ">$fn" or die $!;
        close FILE or die $!;
    }
    

    You'll need to supply your own implementations for the start_new_XFOIL_instance and the output_exists functions, and you'll also want to define your own sets of parameters to pass to XFOIL.