rfurrr

How can I make this code run in parallel ? For loop


I am trying to run this simple for loop as a parallel process as it requires lots of compute power. Any thoughts?

##Load files and libraries---
library(tidyverse)
library(caret)
library(insight)
library(MASS)
library(mfx)
library(furrr)

for (i in 1:16) {
      nested_inter$model[[i]]<- nb_thesis_inter(df= nested_nb$data[[i]], mdl= nested_nb$model[[i]])
      print (paste("Finished model ", i, "out of 16"))
    }

  #nested_inter<- nested_inter %>% 
  #  mutate(model= future_map2(.x= data, .y=model, .f = nb_thesis_inter))

Solution

  • My go to is the future.apply package.

    library(future.apply)
    plan(multisession)
    
    nested_inter$model = future_Map(nb_thesis_inter,
                                    nested_nb$data,
                                    nested_nb$model)
    

    Two things to note.

    1. plan(multisession) allows Windows to be used in parallel. See ?plan for all options.
    2. I did not install all of the packages because the example was not reproducible. The future_Map call may need to be changed to future_map(function (x, y) nb_thesis_inter(df = x, mdl = y), ...) depending on the default argument order of nb_thesis_inter.