I would like to log to console from a worker created using the future package. However, logs from workers go into the void.
For example
library(future)
library(logger)
plan(multisession)
worker_function <- function() {
log_info("Hello from a worker!")
1
}
x <- future_map(1:2, ~ worker_function())
x
Running the above does not show any logs.
I found an issue that seems to be relevant (https://github.com/futureverse/future/issues/306) but not seem to offer a solution.
As a work around I can collect logs from workers and then log them when the main "thread" returns, but this is inconvenient. Any better way?
The default (*) in logger is for log output to be sent to the standard error (stderr), which is impossible to catch and relay in a reliable way using R, including when running in parallel using Futureverse. See <https://github.com/HenrikBengtsson/Wishlist-for-R/issues/55> for details on why. (*) The default is log_appender(append_console), which sends to stderr, although the current document suggests it goes to standard output (stdout) [UPDATE: typo reported https://github.com/daroczig/logger/issues/234].
The simple solution is to set log_appender(append_stdout) on the parallel worker, which you can do by setting it inside your worker_function() .
Verified solution:
library(future)
library(furrr)
library(logger)
plan(multisession)
worker_function <- function() {
log_appender(appender_stdout)
log_info("Hello from a worker!")
1
}
x <- future_map(1:2, ~ worker_function())
x
This will relay the output from logger in your main R session. The output will be relayed as soon as each future is resolved. You should see something like:
INFO [2025-05-09 16:37:55] Hello from a worker!
INFO [2025-05-09 16:37:56] Hello from a worker!
[[1]]
[1] 1
[[2]]
[1] 1
If you want to get near-live logging updates, even while the worker_function() is not yet complete, we have to create a custom appender function that outputs the logged messages as immediateCondition:s . If you want this, please go to Futureverse Discussion forum (<https://github.com/futureverse/future/discussions/>) and we can talk about that over there.