add row and column titles with ggarrange

I have a list of ggplots that may be too complex to arrange using facet_wrap. All plots must share the same legend and should be arranged in a grid. Each column of the grid needs a different title, also each row of the grid needs a differen title.
An absurdly simple example:

library(ggplot2)
library(ggpubr)

plot1<- ggplot() + geom_point(aes(x=1, y=1, col="a"))
plot2<- ggplot() + geom_point(aes(x=1, y=1, col="a"))
plot3<- ggplot() + geom_point(aes(x=1, y=1, col="a"))
plot4<- ggplot() + geom_point(aes(x=1, y=1, col="a"))

plotlist<- list(plot1, plot2, plot3, plot4)

ggarrange(plotlist = plotlist, ncol = 2, nrow = 2, common.legend = TRUE, legend="bottom")

This produces everything needed except the column and row titles, and annotate_figure only adds a global title to the figure. The desired output should look like:

I prefer the patchwork package. Controlling layout is easy. The titles are a bit tricky. Patchwork uses the labs from each plot - so you could add plot titles to the upper plots and maybe add a row to the y label in the left hand side plots.

Arguably easier is to create the titles as plot and add them to your patchwork. You can easily customise your layout like explained in the patchwork vignette

library(ggplot2)
library(patchwork)

plot1<-plot2<-plot3<-plot4<- ggplot() + geom_point(aes(x=1, y=1, col="a"))

row1 <- ggplot() + annotate(geom = 'text', x=1, y=1, label="row 1 title", angle = 90) + theme_void() 

row2 <- ggplot() + annotate(geom = 'text', x=1, y=1, label="row 2 title", angle = 90) + theme_void() 

col1 <- ggplot() + annotate(geom = 'text', x=1, y=1, label="col 1 title") + theme_void() 

col2 <- ggplot() + annotate(geom = 'text', x=1, y=1, label="col 2 title") + theme_void() 

layoutplot <- "
#cccddd
aeeefff
aeeefff
bggghhh
bggghhh
"

plotlist <- list(a = row1, b = row2, c = col1, d = col2, e= plot1, f=plot2, g=plot3, h=plot4)

wrap_plots(plotlist, guides = 'collect', design = layoutplot)

^{Created on 2020-02-22 by the reprex package (v0.3.0)}