purrr::map executes mapping function in different environments for named functions and formula functions

I'm developing a plumber API and need to serialize a custom S3 class. As part of defining a custom serializer, I map over a list of instances of the S3 class. I have implemented a S3 method for as.list so that I can pass objects that jsonlite::toJSON can serialize.

When using purrr::map if I supply the bare function, the dispatch is unable to find the proper method. If I wrap it in a formula/purrr-style function, it works.

My question: Why can't the bare function execution environment find the custom S3 method?

This is a simplified version, but I have the routes saved into separate files which a main plumber.R file mounts.

# plumber.R
library(plumber)
library(jsonlite)
library(purrr)

myEnv <- new.env()

# custom S3 class
myEnv$myLilS3 <- function(x, y) structure(list(x = x, y = y), class = "my_lil_S3")

# S3 Method for my_lil_S3 class
myEnv$as.list.my_lil_S3 <- function(x, ...) unclass(x)

pr("myRoute.R", envir = myEnv)$run(port = 5555)

I'm using a local environment because in the actual use-case, there may be naming conflicts between sourced files.

To illustrate the discrepancy, I have three endpoints:

/function use map with as.list
/formula uses map ~as.list(.x)
/single uses as.list without map

# myRoute.R
myPretties <- map2(1:5, 6:10, myLilS3)

#* @get /function
function(res) {
  res$setHeader("Content-Type", "application/json")

  res$body <- list(values = map(myPretties, as.list)) |>
    toJSON(auto_unbox = TRUE)

  res
}

#* @get /formula
function(res) {
  res$setHeader("Content-Type", "application/json")

  res$body <- list(values = map(myPretties, ~as.list(.x))) |>
    toJSON(auto_unbox = TRUE)

  res
}

#* @get /single
function(res) {
  res$setHeader("Content-Type", "application/json")

  res$body <- list(value = as.list(myPretties[[2]])) |>
    toJSON(auto_unbox = TRUE)

  res
}

The /formula and /single endpoints work as expected, finding the custom S3 method in the myEnv which is used when creating the plumber router.

# GET /forumla
{
    "values": [
        {
            "x": 1,
            "y": 6
        },
        {
            "x": 2,
            "y": 7
        }, 
        ...
}

# GET /single
{
    "value": {
        "x": 2,
        "y": 7
    }
}

/function returns an error because mapping as.list uses the default function, leaving the values with a class of my_lil_s3, which asJSON does not know how to serialize.

# GET /function
{
    "error": "500 - Internal server error",
    "message": "Error: No method asJSON S3 class: my_lil_S3\n"
}

From the as_mapper documentation I know it treats functions and formulas differently. Looking at the mapper functions I would still expect the UseMethod call to be able dispatch properly in both cases.

as_mapper(as.list)
#> function (x, ...) 
#> UseMethod("as.list")
#> <bytecode: 0x000001c01ee2fba8>
#> <environment: namespace:base>

as_mapper(~as.list(.x))
#> <lambda>
#> function (..., .x = ..1, .y = ..2, . = ..1) 
#> as.list(.x)
#> attr(,"class")
#> [1] "rlang_lambda_function" "function"

Solution

Both purrr::map and lapply end up calling compiled code. I didn't dig past that, but it seems that @Ric Villalba's comment explains the essence of why it happens. Inserting some trace code to print call stack tree

 # only trace on the first item (i.e. myPretties[[1]]$x == 1)
  trace(
    as.list, 
    quote(
      if(x$x == 1) walk(sys.parents(), ~print(sys.frame(.x)))
    )
  )

Bare function:

Formula:

If I'm reading the trace correctly, the bare formula gets executed in the <environment:base> where as the formula version gets executed in environment in which the lambda function is defined (i.e. which has the S3 method defined).

Per the suggestion in the similar question, registering the method in the generic dispatch database allows for the bare function to be used with purrr::map.

.S3method("as.list", "my_lil_S3", "as.list.my_lil_S3")