ggplot2: Divide Legend into Two Columns, Each with Its Own Title

I have these factors

require(ggplot2)
names(table(diamonds$cut))
# [1] "Fair"      "Good"      "Very Good" "Premium"   "Ideal" 

which I want to visually divide into two groups in the legend (indicating also the group name):

"First group" -> "Fair", "Good"

and

"Second group" -> "Very Good", "Premium", "Ideal"

Starting with this plot

ggplot(diamonds, aes(color, fill=cut)) + geom_bar() + 
  guides(fill=guide_legend(ncol=2)) +
  theme(legend.position="bottom")

I want to get

(note that "Very Good" slipped in the second column/group)

You can shift the "Very Good" category to the second column of the legend by adding a dummy factor level and setting its colour to white in the legend, so that it can't be seen. In the code below, we add a blank factor level between "Good" and "Very Good", so now we have six levels. Then, we use scale_fill_manual to set the color of this blank level to "white". drop=FALSE forces ggplot to keep the blank level in the legend. There might be a more elegant way to control where ggplot places the legend values, but at least this will get the job done.

diamonds$cut = factor(diamonds$cut, levels=c("Fair","Good"," ","Very Good",
                                             "Premium","Ideal"))

ggplot(diamonds, aes(color, fill=cut)) + geom_bar() + 
  scale_fill_manual(values=c(hcl(seq(15,325,length.out=5), 100, 65)[1:2], 
                             "white",
                             hcl(seq(15,325,length.out=5), 100, 65)[3:5]),
                    drop=FALSE) +
  guides(fill=guide_legend(ncol=2)) +
  theme(legend.position="bottom")

UPDATE: I'm hoping there's a better way to add titles to each group in the legend, but the only option I can come up with for now is to resort to grobs, which always gives me a headache. The code below is adapted from the answer to this SO question. It adds two text grobs, one for each label, but the labels have to be positioned by hand, which is a huge pain. The code for the plot also has to be modified to create more room for the legend. In addition, even though I've turned off clipping for all grobs, the labels are still clipped by the legend grob. You can position the labels outside of the clipped area, but then they're too far from the legend. I'm hoping someone who really knows how to work with grobs can fix this and more generally improve upon the code below (@baptiste, are you out there?).

library(gtable)

p = ggplot(diamonds, aes(color, fill=cut)) + geom_bar() + 
  scale_fill_manual(values=c(hcl(seq(15,325,length.out=5), 100, 65)[1:2], 
                             "white",
                             hcl(seq(15,325,length.out=5), 100, 65)[3:5]),
                    drop=FALSE) +
  guides(fill=guide_legend(ncol=2)) +
  theme(legend.position=c(0.5,-0.26),  
        plot.margin=unit(c(1,1,7,1),"lines")) +
  labs(fill="") 

# Add two text grobs
p = p + annotation_custom(
    grob = textGrob(label = "First\nGroup", 
                    hjust = 0.5, gp = gpar(cex = 0.7)),
    ymin = -2200, ymax = -2200, xmin = 3.45, xmax = 3.45) +
  annotation_custom(
    grob = textGrob(label = "Second\nGroup",
                    hjust = 0.5, gp = gpar(cex = 0.7)),
    ymin = -2200, ymax = -2200, xmin = 4.2, xmax = 4.2)

# Override clipping
gt <- ggplot_gtable(ggplot_build(p))
gt$layout$clip <- "off"
grid.draw(gt)

And here's the result: