[SOLVED] MATLAB Parfor with high number of iterations not starting

MATLAB Parfor with high number of iterations not starting

I am running Matlab 2014a and trying to start a script with parfor. However it gets stuck at

Starting parallel pool (parpool) using the 'local' profile ... connected to 16 workers.

I run a test script before and it worked ok. The difference is that the test script has 4 iterations of parfor and 2 arrays of 5x12x3x4 size which are filled in the parfor loop, whereas the main script has 100 iterations and 2 arrays of 31x12x3x100 size. Any ideas how to make it run? Thanks!

EDIT: here is a simplified version of my code to illustrate the problem, please note that this version, where I don't include the calculations actually works. It is not feasible to include the calculations as they involve other Matlab and a Python script. Also, when I say stuck, I mean that I am not getting any output from the disp(output) line.

close all;
clear all;
clc;

disp('starting');
results_nb = zeros(31,12,3,100);
results_ht = zeros(31,12,3,100);
progress = zeros(1,100);

parfor iter = 1:100

    t = getCurrentTask(); 
    output = ['Worker:' num2str(t.ID) ', Iteration:' num2str(iter)];
    disp(output);
    j_idx=0;
    results_nb_iter=zeros(31,12,3);
    results_ht_iter=zeros(31,12,3);
    for j=[10,20,50]
        j_idx=j_idx+1;
        for i=1:31
            line_nb = zeros(1,12);
            line_ht = zeros(1,12);

            %line_nb = some calculations
            %line_ht = some calculations

            results_nb_iter(i,:,j_idx)=line_nb;
            results_ht_iter(i,:,j_idx)=line_ht;

        end
    end
    results_nb(:,:,:,iter)=results_nb_iter;  
    results_ht(:,:,:,iter)=results_ht_iter;

end

save('results')

exit

Solution

Your deduction is wrong because your measurement is wrong.

disp() inside a parfor loop is not ensured to print to screen exactly when the worker gets to it, it is instead queued and when the worker is free, it will send it to the client to print.

To keep progress of a parfor execution, you need parallel.pool.DataQueue