ggplot: "No non-missing arguments to min/max; returning Inf"

I'm attempting to recreate this plot (my version: lat/lon by year), but keep getting these warnings after running the ggplot code:

> warnings()
Warning messages:
1: `stat_contour()`: Zero contours were generated
2: In min(x) : no non-missing arguments to min; returning Inf
3: In max(x) : no non-missing arguments to max; returning -Inf
...
43: `stat_contour()`: Zero contours were generated
44: In min(x) : no non-missing arguments to min; returning Inf
45: In max(x) : no non-missing arguments to max; returning -Inf
46: Raster pixels are placed at uneven horizontal intervals and will be shifted
ℹ Consider using `geom_tile()` instead.

I thought this solution would remove one of the warnings, but it persists...

> dput(sms2[1:5,])
structure(list(smooth = c("ti(CYR,Longitude,Latitude)", "ti(CYR,Longitude,Latitude)", 
"ti(CYR,Longitude,Latitude)", "ti(CYR,Longitude,Latitude)", "ti(CYR,Longitude,Latitude)"
), type = c("Tensor product int.", "Tensor product int.", "Tensor product int.", 
"Tensor product int.", "Tensor product int."), by = c(NA_character_, 
NA_character_, NA_character_, NA_character_, NA_character_), 
    est = c(0.117025782747128, 0.113479508095921, 0.0933787695695134, 
    0.0895995641580288, 0.0755531029189105), se = c(0.185913143579915, 
    0.181005541009251, 0.152005779645215, 0.142402142342131, 
    0.119622723237741), CYR = c(2008L, 2008L, 2008L, 2008L, 2008L
    ), Longitude = c(-80.3026, -80.3034, -80.3075, -80.3076, 
    -80.31), Latitude = c(25.5694, 25.5684, 25.5618, 25.558, 
    25.5519)), row.names = c(NA, -5L), class = c("smooth_estimates", 
"tbl_df", "tbl", "data.frame"), tensor_term_order = list(`ti(CYR,Longitude,Latitude)` = c("Longitude", 
"Latitude", "CYR")))

est_lim <- c(-1, 1) * max(abs(sms2[["est"]]), na.rm = TRUE)

sms2 |> 
  mutate(fCYR = factor(CYR)) |>
  ggplot(aes(x = Longitude, y = Latitude, fill = est, group = fCYR)) + 
  geom_raster(aes(x = Longitude, y = Latitude, fill = est, group = fCYR)) +
  geom_contour(aes(z = est, group = fCYR, fill = NULL), colour = "black") +
  facet_wrap(~ fCYR) +
  scale_fill_distiller(palette = "RdBu", type = "div") +
  expand_limits(fill = est_lim)

Resultant plot

A start...it seems you need to interpolate to create a full grid per this site: https://community.rstudio.com/t/r-warning-stat-contour-zero-contours-were-generated/111020/2

Here's a simple start:

a <- sms2$Latitude
b <- sms2$Longitude
d <- expand.grid(a,b)
d$z <- rep(sms2$est)
d
ggplot(data = d, aes(Var1, Var2, z = z)) + geom_contour_filled()