I am trying to run R code with multisession parallelism such that all the error messages redirect to the same file. However, the sink()
cannot be created.
library(parallel)
cl <- makePSOCKcluster(2)
f <- function(){
withr::with_message_sink("messages.txt", Sys.sleep(10))
}
clusterCall(cl = cl, fun = f)
## Error in checkForRemoteErrors(lapply(cl, recvResult)) :
## 2 nodes produced errors; first error: Cannot establish message sink when another sink is active.
## Calls: clusterCall -> checkForRemoteErrors
## Execution halted
Given some of the responses, I should elaborate on the purpose of this post. I am developing drake, an R package with multiple parallel backends. Today, I implemented a new hook
argument to make()
, which just wraps individual parallel jobs in a function of the user's choice. What I am really looking for is a hook
that silences the console regardless of parallel backend. Backends in the current development version include
parallel::mclapply()
parallel::parLapply()
base::lapply()
(via parLapply()
for one job)make -j
with a proper Makefile
future::sequential
future::multicore
future::multisession
future.batchtools
backends listed hereI thought I found a hook
that worked for stderr.
hook <- function(){
withr::with_message_sink("messages.txt", Sys.sleep(10))
}
However, withr::with_message_sink()
does not let me sink multiple workers to the same file for the parLapply()
or future::multisession
backends.
Can you just use sink?:
library(parallel)
cl <- makePSOCKcluster(2)
clusterApply(cl, seq_along(cl), function(i) workerID <<- i)
f <- function(){
outtxt <- paste(workerID, "messages.txt", sep="_")
print(outtxt)
sink(outtxt)
Sys.sleep(10)
sink()
}
clusterCall(cl = cl, fun = f)
stopCluster(cl)