How to format api_args in ellmer::chat_gemini to avoid 'Error in !extra_args : invalid argument type' for Gemini Google search grounding?

I am trying to format api_args for ellmer::chat_gemini but each time the call to $chat() would return 'Error in !extra_args : invalid argument type'

The objective is to generate call with Google Search grounding of Gemini.

I tried both direct JSON, and embedded lists:

#this does not work (attempt 1)
api_args <- list(tools = r"--( {"google_search_retrieval": { "dynamic_retrieval_config": {"mode": "MODE_DYNAMIC","dynamic_threshold": 0.3}}} )--" )

#this does not work (attempt 2)
api_args <- list(tools = list(list(google_search_retrieval = list(dynamic_retrieval_config = list(mode = "MODE_DYNAMIC", dynamic_threshold = 0.3)))))

chat_gemini20 <- chat_gemini(system_prompt = system_prompt, model = "gemini-2.0-flash", api_args = api_args, api_key = Sys.getenv("GEMINI_API_KEY"))

gp <- "What is the product identified by isin IE00BYXPSP02?"

chat_gemini20$chat(gp)

Sample REST call I am trying to replicate is straight from Gemini API reference:

echo '{"contents":
          [{"parts": [{"text": "What is the current Google stock price?"}]}],
      "tools": [{"google_search_retrieval": {
                  "dynamic_retrieval_config": {
                    "mode": "MODE_DYNAMIC",
                    "dynamic_threshold": 1,
                }
            }
        }
    ]
}' > request.json

curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent?key=$GEMINI_API_KEY" \
-H "Content-Type: application/json" \
-d  @request.json > response.json

cat response.json

However none of my attempts succeeded to replicate.

Additional information - perhaps the problem is boxing of the scalar values - ["MODE_DYNAMIC"] (attempt 2):

>toJSON(api_args)

{"tools":[{"google_search_retrieval":{"dynamic_retrieval_config":{"mode":["MODE_DYNAMIC"],"dynamic_threshold":[0.3]}}}]}

Solution

ellmer seems to be preprocessing api_args before serializing it to JSON as it ends up in the request as "tools":[{"google_search":null}] instead of "tools":[{"google_search":{}}] :

# API key in GOOGLE_API_KEY env var
library(ellmer)

api_args <- list(tools = list(list(google_search = c())))

# check conversion
jsonlite::toJSON(api_args, pretty = TRUE) |> cat()
#> {
#>   "tools": [
#>     {
#>       "google_search": {}
#>     }
#>   ]
#> }

# test chat with_verbosity level 2 to show request headers and bodies:
httr2::with_verbosity(
  chat_gemini(
    model = "gemini-2.0-flash", 
    api_args = api_args
  )$chat("What is the current Google stock price?"),
  verbosity = 2
)
#> -> POST /v1beta/models/gemini-2.0-flash:streamGenerateContent?alt=sse HTTP/1.1
#> -> Host: generativelanguage.googleapis.com
#> -> User-Agent: httr2/1.1.0 r-curl/6.1.0 libcurl/8.10.1
#> -> Accept: */*
#> -> Accept-Encoding: deflate, gzip
#> -> x-goog-api-key: <REDACTED>
#> -> Content-Type: application/json
#> -> Content-Length: 166
#> -> 
#> >> {"contents":[{"role":"user","parts":[{"text":"What is the current Google stock price?"}]}],"systemInstruction":{"parts":{"text":""}},"tools":[{"google_search":null}]}
#> <- HTTP/1.1 400 Bad Request
#> /../
#> <-
#> Error in `req_perform_connection()`:
#> ! HTTP 400 Bad Request.
#> • Request contains an invalid argument.

Though something like
list(tools = list(list(google_search = structure(list(), names = character(0))))) or just
jsonlite::parse_json('{"tools": [{ "google_search": {} }] }')
does work:

api_args <- jsonlite::parse_json('{"tools": [{ "google_search": {} }] }') 

chat <- chat_gemini(model = "gemini-2.0-flash", api_args = api_args)
chat$chat("What is the current Google stock price?")
#> As of February 12, 2025, the current price of Alphabet Inc (Google) Class C 
#> stock (GOOG) is $185.37. It has decreased by -0.60% in the past 24 hours.

While it's a valid grounded response and you can take a closer look by calling it through httr2::with_verbosity(), it seems that ellmer currently does not collect nor provide access to grounding data.