Force keys size to be smaller in multiple lines legend in ggplot

Is there any way to force legend key sizes to be independent from line height when we have legend text with multiple lines? I tried legend.key.height, ggtext::element_textbox_simple, and guides but none of them help to resize key legends independent from line height.

What I want is resize legend keys to be same size as the small grey ones while having my legend text exactly as it is.

Following the second approach by @teunbrand you could (probably) achieve your desired result by passing the custom draw key function by @teunbrand to the key_glyph argument of geom_col.

Using a minimal reproducible example based on some fake data:

library(ggplot2)
library(grid)
library(rlang)

set.seed(1)

draw_square <- function(data, params, size) {
  if (is.null(data$size)) {
    data$size <- 0.5
  }
  lwd <- min(data$size, min(size) / 4)
  grid::rectGrob(
    width = unit(1, "snpc") - unit(lwd, "mm"),
    height = unit(1, "snpc") - unit(lwd, "mm"),
    gp = gpar(
      col = data$colour %||% NA,
      fill = alpha(data$fill %||% "grey20", data$alpha),
      lty = data$linetype %||% 1,
      lwd = lwd * .pt,
      linejoin = params$linejoin %||% "mitre",
      lineend = if (identical(params$linejoin, "round")) "round" else "square"
    )
  )
}

# create the dataframe
df <- data.frame(
  x = 1:10,
  y = LETTERS[1:10],
  fill = sample(
    c(
      "Higher for men than for women",
      "No significant difference",
      "Higher for women than for men"
    ),
    10,
    replace = TRUE
  )
)

df$fill <- factor(df$fill, c(
  "Higher for men than for women",
  "No significant difference",
  "Higher for women than for men"
))

ggplot(df, aes(x = -x, y = y, fill = fill)) +
  geom_col(
    key_glyph = draw_square, width = .8
  ) +
  scale_fill_manual(
    values = c("#5C3B73", "grey", "#974156"),
    labels = label_wrap_gen(28)
  ) +
  theme_void(
    base_size = 20,
    base_family = "Roboto"
  ) +
  theme(
    legend.key.spacing.y = unit(20, "pt"),
    plot.margin = unit(rep(5.5, 4), "pt"),
    plot.background = element_rect(fill = "#EDF6F9"),
    legend.justification = c(1, .95)
  ) +
  labs(fill = NULL)