How should I position a y-axis title in the top plot margin, left-justified?

In R, the default placement and orientation of the y-axis title is vertical (angle = 90) and in the left margin of the plotting area:

ggplot(iris, aes(x = Species, y = Sepal.Length, fill = Species)) + geom_boxplot() 

For various design reasons, I would prefer my y axis titles to be horizontal (angle = 0) and in the top margin of the plotting area, left-justified so as to be even with the left edge of the y-axis labels (see images below).

I have so far found two ways to achieve this, but both are hacky. The first is to use trial-and-error margin adjustments to achieve this:

ggplot(iris, aes(x = Species, y = Sepal.Length, fill = Species)) + 
  geom_boxplot() +
  theme(axis.title.y = element_text(margin = margin(r = -64), vjust = 1.075, angle = 0),
        
        plot.margin = margin(t = 35, l = 10, b = 10, 10))

The second is to supplant the y-axis title with a subtitle:

ggplot(iris, aes(x = Species, y = Sepal.Length, fill = Species)) + 
  geom_boxplot() +
  theme(axis.title.y = element_blank(),
        plot.subtitle = element_text(hjust = -0.03),
        plot.title = element_blank()
        ) +
  ggtitle("", subtitle = "Sepal.Length")

This second way interacts with top-positioned legends in an undesirable way, so I prefer the former currently.

In both cases, to figure out what the exact vjust/hjust and margin adjustment values need to be is non-programmatic. It'd be much nicer to do this programmatically! Ideally, there'd be a base-ggplot2 way, but if there isn't, that's ok.

This question seems related but doesn't provide a satisfactory solution. To clarify, I want the title outside the plotting area, not inside of it, and even with the left edges of the y axis labels, not with the y axis line.

Edit: A third hack, as noted in Jon Spring's answer, is to use annotations with clipping off:

ggplot(iris, aes(x = Species, y = Sepal.Length, fill = Species)) + 
  geom_boxplot() +
  theme(axis.title.y = element_blank(),
        plot.margin = margin(t=25)
  ) +
  coord_cartesian(clip = "off") +
  annotate("text", x = -Inf, y = Inf, label = "Sepal.Length", hjust = .15, vjust = -1)

But this isn't less hack-y or more programmatic, unfortunately.

It is possible to do this automatically without tweaking the label position every time or clashing with the title / captions.

This method relies on the fact that when a ggplot is drawn, it is first converted into a gtable, which is an unevenly-spaced grid of graphical objects. As it happens, there is an empty cell in this table directly above the y axis which has zero height unless you have an x axis along the top of your plot. You can simply give this cell some height and draw a left-aligned textGrob in it. By default, clipping is turned off in this cell, so the text can stretch out into the area above the plot.

We can do all this inside a function that we pass the plot to:

switch_axis_label <- function(p) {
  
  lab <- p$scales$get_scales("y")$name
  if(is.null(lab)) lab <- p$labels$y
  
  size <- calc_element("axis.title.y", p$theme)$size
  if(is.null(size)) calc_element("axis.title.y", theme_get())$size
  
  if(!is.null(p$scales$get_scales("y"))) {
    y <- which(unlist(lapply(p$scales$scales, \(x) "y" %in% x$aesthetics)))
    p$scales$scales[[y]]$name <- NULL
  }

  p <- p + labs(y = NULL)
  gt <- p |> ggplot_build() |> ggplot_gtable()
  
  gt$grobs[[2]] <- grid::textGrob(lab, x = 0, y = 0.5, hjust = 0,
                                  gp = grid::gpar(fontsize = size))
  
  gt$heights[8] <- grid::unit(3, "lines")
  
  grid::grid.draw(gt)
}

Testing, we get the following

library(ggplot2)

p <- ggplot(iris, aes(x = Species, y = Sepal.Length, fill = Species)) + 
  geom_boxplot() 

switch_axis_label(p)

This may be all you need for occasional use.

However, since this function outputs a gtable rather than a ggplot object which we can further tweak, it needs to be called as a final step in drawing your plot. This isn't ideal and runs against normal ggplot design philosophy. To get round this we can write a ggplot_gtable method that is a wrapper for this function. Unfortunately, this means we need to also write simple ggplot_build and ggplot_add methods.

switch_y_axis <- function() {
  structure(list(), class = "axis_switcher")
}

ggplot_add.axis_switcher <- function(object, plot, name = "switcher") {
  class(plot) <- c("switcher", class(plot))
  return(plot)
}

ggplot_build.switcher <- function(plot) {
  class(plot) <- c("gg", "ggplot")
  output <- ggplot_build(plot)
  class(output) <- c("switched", class(output))
  output
}

ggplot_gtable.switched <- function(plot) {
  switch_axis_label(plot$plot)
}

Now we can add + switch_y_axis() to our plot any time we want the axis title moved to the top of the y axis:

p + theme(legend.position = "top") + switch_y_axis()

And there is always room to draw the labels:

p + switch_y_axis() + ggtitle("Thank goodness", "The labels don't clash")

And, to show generality,

ggplot(mtcars, aes(wt, mpg)) + geom_point() + switch_y_axis()

and, further to comments, it works with scale_y_ functions:

ggplot(iris, aes(x = Species, y = Sepal.Length, fill = Species)) + 
  geom_boxplot() + 
  scale_y_continuous("I have a new title now") +
  switch_y_axis()

There are some caveats. Firstly, if you have an x axis at the top of your plot, this will clash with it. Presumably having your y axis title here would clash or look messy against a top x axis however you drew it, so I don't see this as a major concern.

Secondly, there is no guarantee that ggplot's internal implementation won't change in the future, and this function would stop working if it did. That seems unlikely at this point.

Thirdly, the way the function is written currently, you cannot set theme options for the text label other than its size. If you wanted to be able to control other aspects of the label's style as normal through theme, you would need to strip the various theme options using theme_get and apply them to the text label via grid::gpar as I have done with size in the function.