Overlaying discrete and continuous versions of the same variable in ggplot

I have a dot graph where points defined by colours on a continuous colour scale based on the value of one variable, bout I also want to graph grouped smooth lines over the top of this dot graph. The groups for the smooth lines will be discretized versions of the variable that the continuous dot graph is based on.

Toy data

df <- mtcars %>%
        mutate(mpg_cat = factor(case_when(mpg > 10 & mpg <= 14 ~ "low",
                                          mpg > 14 & mpg <= 19 ~ "mid",
                                          mpg > 19 ~ "high"),
                                levels = c("low", "mid", "high"))) %>%
          select(mpg, mpg_cat, disp) 

In isolation the point graph works fine

ggplot() +
       geom_point(data = df,
                  mapping = aes(x = disp,
                                y = mpg,
                                colour = mpg)) +
       scale_colour_distiller(palette = "Spectral")

As does the smooth graph

ggplot() +
       geom_smooth(data = df,
                   mapping = aes(x = disp,
                                 y = mpg,
                                 colour = mpg_cat),
                   method = "loess",
                   se = FALSE) +
       scale_colour_brewer(palette = "Spectral")

But when I try to combine the two

ggplot() +
       geom_point(data = df,
                  mapping = aes(x = disp,
                                y = mpg,
                                colour = mpg)) +
       geom_smooth(data = df,
                   mapping = aes(x = disp,
                                 y = mpg,
                                 colour = mpg_cat),
                   method = "loess",
                   se = F)

I get the error

Error in `scale_colour_continuous()`:
! Discrete values supplied to continuous scale.
ℹ Example values: low, low, low, low, and low
Run `rlang::last_trace()` to see where the error occurred.

Can anyone give me some guidance here?

Implementing @Kat suggestion, and reversing the category orders to match worked for me:

ggplot() +
  geom_point(data = df,
             mapping = aes(x = disp,
                           y = mpg,
                           colour = mpg)) +
  scale_colour_distiller(palette = "Spectral") +
  ggnewscale::new_scale_color() +
  geom_smooth(data = df,
              mapping = aes(x = disp,
                            y = mpg,
                            colour = mpg_cat |> forcats::fct_rev()),
              method = "loess",
              se = F) +
  scale_colour_brewer(palette = "Spectral", name = NULL)