rerror-handlingparallel-processingmultiprocessingparallel-foreach

"Error in checkForRemoteErrors(val) : 2 nodes produced errors; first error: could not find function "wincrqa"

I am currently trying to run a parallelized RQA with the following code.

library(snow)
library(doSNOW)
library(crqa)
    my_wincrqa = function(x, y){
      wincrqa(x, y, windowstep = 1000, windowsize = 2000,
              radius = .2, delay = 4, embed = 2, rescale = 0, normalize = 0,
              mindiagline = 2, minvertline = 2, tw = 0, whiteline = F,
              side = "both", method = "crqa", metric = "euclidean", datatype = "continuous")
    }
cl<-makeCluster(11,type="SOCK")
start_time <- Sys.time()
WCRQA_list = clusterMap(cl, my_wincrqa, HR_list, RR_list)
end_time <- Sys.time()
end_time - start_time

Unfortunately, I get this: "

Error in checkForRemoteErrors(val) : 2 nodes produced errors; first error: could not find function "wincrqa"

I know there is probably sum error in setting up the parallel processing, but I am not able to resolve it. I also tried a similar thing using the parallel() package.

I am happy for any help!

Best, Johnson

Solution

  • The issue is that you’ve loaded and attached the ‘crqa’ package in your main execution environment, but the cluster nodes are running code in separate, isolated R sessions — they don’t see the same loaded packages or global variables!

    The easiest solution is to replace use of wincrqa with a fully qualified name, i.e. to use crqa::wincrqa inside your function.

    Alternatively, it is possible to attach the ‘crqa’ package on all cluster nodes prior to executing the function:

    clusterEvalQ(cl, library(crqa))
WCRQA_list = clusterMap(cl, my_wincrqa, HR_list, RR_list)