osmdata_sf returns failed to perform HTTP request curl::curl_fetch_memory() error in R?

I am trying to query osm data in R using the following code:

library(osmdata)

bb <- getbb('North Goa, India')

q <- opq(bbox = bb, osm_types = c("node", "way", "lines")) %>%
  add_osm_feature(key = 'highway') %>%
  osmdata_sf()

which returns me following error

Error in httr2::req_perform(): ! Failed to perform HTTP request. Caused by error in curl::curl_fetch_memory(): ! Could not resolve hostname [overpass.kumi.systems]: Could not resolve host: overpass.kumi.systems

Backtrace:
     ▆
  1. ├─... %>% trim_osmdata(bb)
  2. ├─osmdata::trim_osmdata(., bb)
  3. │ └─methods::is(dat, "osmdata_sf")
  4. └─osmdata::osmdata_sf(.)
  5.   └─osmdata:::fill_overpass_data(obj, doc, quiet = quiet)
  6.     └─osmdata:::overpass_query(...)
  7.       └─osmdata::overpass_status(quiet)
  8.         └─httr2::req_perform(req)
  9.           └─base::tryCatch(...)
 10.             └─base (local) tryCatchList(expr, classes, parentenv, handlers)
 11.               └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
 12.                 └─value[[3L]](cond)

Solution

Try fixing the API URL to one of the endpoints listed at https://wiki.openstreetmap.org/wiki/Overpass_API#Public_Overpass_API_instances :

library(osmdata)
#> Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright
set_overpass_url("https://overpass-api.de/api/interpreter")

bb <- getbb('North Goa, India')
q <- opq(bbox = bb, osm_types = c("node", "way", "lines")) %>%
  add_osm_feature(key = 'highway') %>%
  osmdata_sf(quiet = FALSE)
#> Issuing query to Overpass API ...
#> Announced endpoint: lambert.openstreetmap.de/
#> Query complete!
#> converting OSM data to sf format
q
#> Object of class 'osmdata' with:
#>                  $bbox : 15.4080924,73.6756012,15.8007631,74.2823624
#>         $overpass_call : The call submitted to the overpass API
#>                  $meta : metadata including timestamp and version numbers
#>            $osm_points : 'sf' Simple Features Collection with 164882 points
#>             $osm_lines : 'sf' Simple Features Collection with 19914 linestrings
#>          $osm_polygons : 'sf' Simple Features Collection with 134 polygons
#>        $osm_multilines : NULL
#>     $osm_multipolygons : NULL

Note that this issue is somewhat difficult if not impossible to properly reproduce -- osmdata picks a random API endpoint during package load, currently there are 2 choices and 50/50 chance to end up with overpass.kumi.systems:

osmdata:::.onLoad
#> function (libname, pkgname) 
#> {
#>     op <- options()
#>     available_apis <- c("https://overpass-api.de/api/interpreter", 
#>         "https://overpass.kumi.systems/api/interpreter")
#>     op.osmdata <- list(osmdata.base_url = sample(available_apis, 
#>         1))
#>     toset <- !(names(op.osmdata) %in% names(op))
#>     if (any(toset)) {
#>         options(op.osmdata[toset])
#>     }
#>     invisible()
#> }
#> <bytecode: 0x0000019c4dd1b348>
#> <environment: namespace:osmdata>

# trigger osmdata:::.onLoad and api url choices in clean R environments
f <- function(seed){ 
  set.seed(seed)
  osmdata::get_overpass_url()
}

for (seed in c(1,5)){
  callr::r(f, args = list(seed = seed)) |> print() 
}
#> [1] "https://overpass-api.de/api/interpreter"
#> [1] "https://overpass.kumi.systems/api/interpreter"

For most people / most of the time overpass.kumi.systems probably works just fine, but it does seem to have some intermittent DNS issues depending on location / service provider / your own DNS config (service provider resolver vs some public DNS service like 1.1.1.1 or 8.8.8.8, caching DNS service in your local network, ...); and if 1st name resolve fails or just times out, following requests are likely to succeed.

For larger OSM bulk imports please consider services like Geofabrik's OSM Data Extracts instead of Overpass API, conveniently accessible through {osmextract} package. E.g. something like this to work with asia/india/western-zone and create a pre-filtered transportation network clipped to admin. boundary:

library(osmextract)

# set set a persistent download directory
Sys.setenv(OSMEXT_DOWNLOAD_DIRECTORY = "D:/geofabrik/")

# extract admin. boundary 
ng_bound <- oe_get_boundary(place = "North Goa, India", name = "North Goa")

# extract transport network, clipped to buffered ng_bound
ng_trnet <- 
  oe_get_network(
    place = "North Goa, India", 
    mode = c("cycling", "driving", "walking"), 
    boundary = sf::st_buffer(ng_bound, 1000),
    boundary_type = "clipsrc"
  )