I'm using doParallel to do fairly long parallel processing with foreach. Rather than most examples I see, where a computationally-intensive but input-light code is fed into the loop, I'm using foreach to coordinate the simultaneous processing of a number of large, independent datasets. So inside the loop, I'm using metadata to read in a file from disk, operate on it, and write back out.
Before I turned this operation into a foreach loop, I was writing out debug messages using messages(). However, since I've switched to using foreach and %dopar%, I've noticed that the loop 'goes dark': it's doing what it ought to, but I'm not receiving any output. (I should mention that this loop is written into a script that I'm calling from the shell with Rscript.)
I'm guessing that this has something to do with the fact that doParallel spins off other threads—maybe those threads no longer know where to dump standard output? Thoughts?
If you want to output from a parallel-foreach loop, just use the option outfile: makeCluster(no_cores, outfile = "/path/to/log_file.txt").
Note the the logs of all workers are written to the same file (in the order in which they arrive).