In the MATLAB documentation we have a code example for the parfeval
function. I have some questions about it. This is the code:
p = gcp();
%// To request multiple evaluations, use a loop.
for idx = 1:10
f(idx) = parfeval(p,@magic,1,idx); % Square size determined by idx
end
%// Collect the results as they become available.
magicResults = cell(1,10);
for idx = 1:10
%// fetchNext blocks until next results are available.
[completedIdx,value] = fetchNext(f);
magicResults{completedIdx} = value;
fprintf('Got result with index: %d.\n', completedIdx);
end
parfeval
work? magic
function to a specific worker?If we'd only have one line of code f = parfeval(p,@magic,1,10);
:
parfeval
and parfevalOnAll
?fetchNext
and fetchOutputs
? fetchNext
after the first for
loop and a second loop to get the results? I think we can use magicResults{idx} = fetchOutputs(f(idx));
in the second loop. We have the same results. What is difference between these two structures?
- How does
parfeval
work?- Does this function send every (idx) evaluational of the
magic
function to a specific worker?
From the documentation: parfeval
requests asynchronous execution on a worker in a parallel pool. You can cancel the execution with cancel
. It just sends it to any free worker. Use parfevalOnAll
to execute a function on all workers. What parfeval
does is providing the communication framework (parallel.FevalFuture
) to find out if a the function evaluation is finished and what were the results.
- If we'd only have one line of code
f = parfeval(p,@magic,1,10);
:
- How would this code work?
- Will it send evaluation only to one worker and returns the output?
- What is the difference between
parfeval
andparfevalOnAll
?
It would send evaluation to only one worker (the next free one I guess) and you can fetch the output by calling fetchOutputs
(will wait for finished before). parfevalOnAll
however executes the function on all workers - there are probably not very many useful usecases for this. Use parfeval
for most/all of your computation jobs.
- What is the difference between
fetchNext
andfetchOutputs
?- Why do we need
fetchNext
after the firstfor
loop and a second loop to get the results?- We don't have all results after first loop? Why?
- I think we are waiting for workers to complete the process in second loop. Is this true? Can't we do it without any loop?
fetchNext
waits until any one of the function evaluations in a list of FEvalFutures
is finished and returns the index and the result. fetchOutputs
just works on a single FEvalFuture
and waits for this and returns the result only. So use fetchNext
if you have scheduled several parallel function evaluations and fetchOutputs
if there was only one.
Why a loop? Well you have scheduled 10 evaluations, so you should wait for the results of these 10. You know you must call fetchNext
10 times, therefore use a loop. In order not to wait too long you use fetchNext
which comes back as early as possible (when there is at least one result), but you still have to call it 10 times. You could do with a while loop instead but a loop it will be. The for loop from the example is adequate.
- I think we can use
magicResults{idx} = fetchOutputs(f(idx));
in the second loop. We have the same results. What is difference between these two structures?
Yes, you could use it, but then you would very probably wait longer until the first result is available to you. This is usually inefficient.
The reason is that the results arrive asynchroneously (as they are executed asynchroneously). So why waiting for the first one, if the second one is already available (and could be further processed maybe..). Better wait for the next one available using fetchNext
.
I hope it got clear how Matlab tries to deal with asynchronicity. They basically have a function fetchNext
which returns as soon as at least one of a bunch of asynchronous operations is finished. This is usually a very efficient design pattern. I strongly recommend to use it.