transition_time with shadow mark for only one geom / layer

I believe the images below should be fairly self explanatory. When using transition_time, does anyone have an idea how to create a shadow mark for only one geom (for labelling)?

Any workarounds very welcome. In a normal world, I'd create two plots (the plot and the title), but I don't know how to use gganimate on such a combined plot, at least I don't find a solution with cowplot and patchwork (given that at least patchwork shares the same developer, I thought it could maybe work.)

library(ggplot2)
library(gganimate)

myfoo <- data.frame(time = 1:100, x = 1:100, y = 1)

p <- 
  ggplot(myfoo) +
  geom_point(aes(x, y)) +
  geom_text(aes(x = 0, y = 0, label = time))

p_anim <- 
  p +
  transition_time(time) +
  shadow_mark()

animate(p_anim, height = 150, width = 250)

The label in the bottom left should not have a shadow mark.

What I am looking for is something like the following:

p_title <- 
  ggplot(myfoo) +
  geom_point(aes(x, y)) +
  labs(title = "{frame_time}")

p_title_anim <- 
  p_title +
  transition_time(time) +
  shadow_mark()

animate(p_title_anim, height = 150, width = 250)

The interactive element from the title should be a plot annotation.

As suggested by Z.Lin in her comment, here the solution with the exclude_layer argument in shadow_mark():

library(ggplot2)
library(gganimate)

myfoo <- data.frame(time = 1:100, x = 1:100, y = 1)

p <- 
  ggplot(myfoo) +
  geom_point(aes(x, y)) +
  geom_text(aes(x = 0, y = 0, label = time))

p_anim <- 
  p +
  transition_time(time) +
  shadow_mark(exclude_layer = 2)

animate(p_anim, height = 150, width = 250)

^{Created on 2020-10-03 by the reprex package (v0.3.0)}