pythondistributed-computingipython-parallel

Specify a number of ipengine instances to be launched within ipyparallel cluster

Speaking of ipyparallel, is it possible to specify a number of ipengines to simultaneously launch on a slave machine, and if so - how do I do it?

For example, one can specify a number of engines to launch on localhost with ipcluster start -n command, i.e. $ ipcluster start -n 4, but -n flag does not work with ipengine command.

Solution

  • The docs explicitly state that you need to "repeatedly" call ipengine:

    Next, start however many instances of the engine you want using (repeatedly) the command:

    $ ipengine

    To start 4 engines from the "slave machine" use this at the command line:

    for engine in {1..4}
do
  ipengine &
done

    It's probably worth also reading starting the controller and engines on different hosts.