rmultidplyrcallr

How to set time out in multidplyr

I inconsistently get following error when using multidplyr (ie, for the same data sometimes I get the error, sometimes not):

Error in rs_init(self, private, super, options, wait, wait_timeout) : Could not start R session, timed out

My setting looks like:

list1 <- list() 

for (i in 1:500){
  while(TRUE){
    test1 <- try(df %>% 
                   group_by(id) %>% 
                   recursive_func_c())
    if(!is(test1, 'try-error')) break
  }
  list1[[i]] <- test1
}

Where recursive_func_c is a function that calls:

cluster <- multidplyr::new_cluster(7)

As far as I understand it correctly, a multidplyr cluster consists of multiple R processes created by callr. It seems the error message I get comes from the callr package. For debugging, I would like to set a longer time out time for callr within multidplyr. Is that possible? Thank you for any hint.

Solution

  • Think this issue was solved here: https://github.com/tidyverse/multidplyr/issues/97

    15 seconds is propably too short for my setting.

    If you face the same issue, you can try to temporarily change wait_timeout of new_cluster() with trace(new_cluster, edit=TRUE) (more regarding general function modification can be found here: Modify package function)

    60 seconds seems to work for me.