rcran

R CRAN Check fail when using parallel functions

I want to submit a package to CRAN that uses parallel computation using parallel::makeCluster(parallel::detectCores()).

When I build the package everything works fine, but when I check the package (devtools::check(document = FALSE)) it returns the error:

Running examples in ‘TESTER-Ex.R’ failed
The error most likely occurred in:

> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: hello_world
> ### Title: Prints hello world
> ### Aliases: hello_world
> 
> ### ** Examples
> 
> hello_world()
Error in .check_ncores(length(names)) : 8 simultaneous processes spawned
Calls: hello_world -> <Anonymous> -> makePSOCKcluster -> .check_ncores
Execution halted

I recreated the error in an MWE-package (TESTER), which only has one function hello_world, I have uploaded the whole TESTER-package to GitHub but the it should be reproducible from the following function.

#' Prints hello world
#'
#' @return nothing
#' @export
#'
#' @examples
#' hello_world()
hello_world <- function() {

  # initiate cluster
  cl <- parallel::makeCluster(parallel::detectCores())

  # stop cluster
  parallel::stopCluster(cl)

  cat("Hello World\n")
  return(invisible(NULL))
}

I have looked through Writing R Extensions but couldn't find anything related to this issue, neither could I find a question on SO.

Any ideas what causes this error and how to resolve it?

Solution

  • CRAN limits the number of cores available to packages to 2, for performance reasons. From the CRAN policies document:

    If running a package uses multiple threads/cores it must never use more than two simultaneously: the check farm is a shared resource and will typically be running many checks simultaneously.

    I use something like the following in my tests:

    chk <- Sys.getenv("_R_CHECK_LIMIT_CORES_", "")

if (nzchar(chk) && chk == "TRUE") {
    # use 2 cores in CRAN/Travis/AppVeyor
    num_workers <- 2L
} else {
    # use all cores in devtools::test()
    num_workers <- parallel::detectCores()
}