x-axis text disappears after customizing ggpairs plot

I want to create a correlation matrix with ggpairs. I only need the lower part. The upper and diagonal parts are not relevant. Furthermore, I want to keep the final plot simple and therefore remove the axis labels at the top left and bottom right, where there are no more plots anyway (see below). I succeed in doing this, but my approach leads to the x-axis labelling also disappearing afterwards and I do not know why and how I can restore it.

My question therefore: How can I restore the x-axis labelling or is there another approach how I can produce the desired plot?

library(tidyverse)
library(GGally)

# Function to customize single plots within ggpairs
lowerFn <- function(data, mapping, ...) {
  p <- ggplot(data = data, mapping = mapping) +
    geom_point(alpha = 0.8, size = 1) +
    geom_smooth(color = "red", method = 'lm', linewidth = 0.8, se = FALSE,...)
  p
}

# Create ggpairs plot
ggtest <- ggpairs(mtcars,
               column = 4:7,
               lower = list(continuous = wrap(lowerFn)),
               switch = "both",
               upper = list(continuous = "blank"),
               diag = list(continuous = "blankDiag"),
)
ggtest

# Customize ggpairs plot
ggtest$yAxisLabels = ggtest$yAxisLabels[2:4]
ggtest$xAxisLabels = ggtest$xAxisLabels[1:3]
ggtest$plots = ggtest$plots[c(5:7,9:11,13:15)]
ggtest

# x-axis labelling is missing in the final ggpairs plot

The output of ggpairs isn't supposed to be wrangled like this, as there are several interdependent data members of the final object which are used in plot creation. What you are doing is fairly hacky.

However, if you want this particular hack to work, you need to also update the nrow and ncol members of ggtest

Original

ggtest

Hacked:

ggtest$yAxisLabels = ggtest$yAxisLabels[2:4]
ggtest$xAxisLabels = ggtest$xAxisLabels[1:3]
ggtest$plots = ggtest$plots[c(5:7, 9:11, 13:15)]
ggtest$nrow <- 3
ggtest$ncol <- 3
ggtest