How to use svg with geom_image in R?

I like to use some icons and images in my ggplot, thus use the ggimage package. I can make it work with local or online images in .png format. But it does not work with svg such as https://raw.githubusercontent.com/ionic-team/ionicons/main/src/svg/hourglass.svg Initially I also was trying a geom_icon() layer.

library("ggplot2")
library("ggimage")

mtcars %>% 
  mutate(am = factor(am)) %>% 
  group_by(am) %>% 
  summarise(avg_mpg = mean(mpg)) %>% 
  mutate(image = c("https://www.r-project.org/logo/Rlogo.png",
              "https://jeroenooms.github.io/images/frink.png")) %>% 
  ggplot(aes(x=factor(am), y=avg_mpg)) + 
  geom_col(width = 0.5) + 
  geom_image(aes(image=image), size=.2) + 
  theme_minimal()

Install rsvg package with all dependencies.