I'm interested in learning more about how furrr
finds stuff from the global environment, and asked generally about the black magic it performs. Here's a specific example of a behavior I didn't understand and could use some help with: What do I need to change in either in the future_map
call or in the get
call to return "C"
and "F"
?
# load furrr, describe "plan"
library(furrr)
nc<-2
plan(strategy = multiprocess, workers = nc)
# create objects
a<-list("A", "B", "C")
b<-list("D", "E", "F")
#works fine
future_map(1:5, function(foo){
map(c("a", "b"), function(my_object_name){
bar<-my_object_name
print(bar)
})
})
# object 'a' not found
future_map(1:5, function(foo){
map(c("a", "b"), function(my_object_name){
bar<-get(my_object_name)[[3]]
print(bar)
})
})
EDIT
It seems like this issue is not reproducible on all systems and may have to do with my installation of furrr.
Despite the warning the package gives about multicore plans, this is an issue with multiprocess
and multisession
but not plan(strategy=multicore,...
.
I think I tripped across some known weird behavior with the future
package with documented workarounds. See vignette in future documentation.
To add variables to the exported globals, use the globals
argument in future
, which translates in furrr
to , .options = future_options(globals(structure=T, add="missing_object"
I suspect this might have also been one of my problems:
...The above error occurs because, contrary to the master R process, the R worker that evaluated the
future
expression does not havedata.table
loaded. Instead the evaluation falls back to the [.data.frame method, which is not what we want.Until the future framework manages to identify
data.table
as a required package (which is the goal), we can guide future by specifying additional packages needed...