For some reason I cannot get IPython parallel function DirectView.apply()
to really parallelize a function call. DirectView.map()
works as expected. Am I doing something wrong here? A working example script
import time
from datetime import datetime
from ipyparallel import Client, require
@require(time)
def wait(seconds=1):
time.sleep(seconds)
if __name__=='__main__':
client = Client()
print('engine ids: {}'.format(client.ids))
dview = client.direct_view((0, 1, 2, 3))
dview.block = False
print('view targets: {}'.format(dview.targets))
print('dview.apply...')
t0 = datetime.now()
results = [dview.apply(wait) for i in range(4)]
while len(results) > 0:
results.pop(0).get()
print('time: {}'.format(datetime.now() - t0))
print('dview.map... ')
t0 = datetime.now()
results = dview.map(wait, [1]*4)
print('time: {}'.format(datetime.now() - t0))
prints
engine ids: [0, 1, 2, 3]
view targets: [0, 1, 2, 3]
dview.apply...
time: 0:00:04.021680
dview.map...
time: 0:00:01.013941
showing that apply
clearly does not perform as I expected.
My system is Ubuntu 14.04, Python 3.4.3, IPython 4.2.0.
Seems that I misinterpreted the documentation. Here it says that apply(f, *args, **kwargs)
calls f(*args, **kwargs)
on remote engines, returning the result.
If I understand correctly, this means that the function is applied on all of the engines, instead of running it once on just one of them.