Create line across multiple plots in ggplot2

I want to create a horizontal line going across two plots combined with the patchwork package.

library(ggplot2)
library(patchwork)

# Annotation after plot
p1 <- ggplot(mtcars, aes(x=disp,y=mpg))+
  geom_point()
p2 <- ggplot(mtcars, aes(x=hp,y=mpg))+
  geom_point()
# Want line across plots at y (mpg) of 15
p3 <- (p1+p2)+annotate("segment",x=-Inf,xend=Inf,y=15,yend=15)
p3

This method only puts the line across the last plot (p2).

Trying with putting the annotation with each plot.

# Annotation with each plot
p1 <- ggplot(mtcars, aes(x=disp,y=mpg))+
  geom_point()+
  annotate("segment",x=-Inf,xend=Inf,y=15,yend=15)
p2 <- ggplot(mtcars, aes(x=hp,y=mpg))+
  geom_point()+
  annotate("segment",x=-Inf,xend=Inf,y=15,yend=15)

p1+p2

This method puts the line on each plot but not between.

I want something like this:

You can draw the line using grid.draw, which plots the line over whatever else is in the plotting window:

library(grid)
p3
grid.draw(linesGrob(x = unit(c(0.06, 0.98), "npc"), y = unit(c(0.277, 0.277), "npc")))

However, there are a couple of caveats here. The exact positioning of the line is up to you, and although the positioning can be done programatically if this is something you are going to do frequently, for a one-off it is quicker to just tweak the x and y values to get the line where you want it, as I did here in under a minute.

The second caveat is that the line is positioned in npc space, while ggplot uses a combination of fixed and flexible spacings. The upshot of this is that the line will move relative to the plot whenever the plot is resized. Again, this can be fixed programatically. If you really want to open that can of worms, you can see a solution to doing something similar with points in my answer to this question