Move ggplot legend closer to plot

Below is some code to show plotting of a scatterplot 2 different ways.

The first way gets me exactly what I want to see, but if I were to make it into a template for a whole bunch of scatterplots, I'd still have to play around with the plot.margin variable to ensure that the legend is still within the image's margins.

The second uses a different method to get the legend into the position I desire. However, I can't move the legend closer to the plot without messing up the box around the legend, as shown in the third plot taking the suggestion from here and here that obviously doesn't work here. That suggestion moves the text, but not the legends box!

library(ggplot2)

data_points = data.frame(x=rnorm(n=16), 
                         y=rnorm(n=16), 
                         col=c(rep("red",8), rep("blue",8)),
                         shap=rep(c(rep("sq",4), rep("tr",4)), 2))

desired_look = theme(plot.margin = margin(r = 25), # I need to play with the margin value or else it will be cut off
                         legend.justification = c("left", "top"),
                         legend.position = c(1,1)
                         #legend.margin = margin(c(1,1,1,3)))

just =   theme(legend.justification = "top") # Way too far away from the plot, how do I get it closer?

just_box_probs =   theme(legend.justification = "top", 
                          legend.box.margin = margin(l = -10)) # text is now outside the box!!!

p = ggplot(data_points, aes(x=x, y=y, color=col, shape=shap)) + geom_point() + theme_classic() +
    scale_x_continuous(sec.axis = dup_axis(labels = NULL)) +
    scale_y_continuous(sec.axis = dup_axis(labels = NULL)) + 
    theme(legend.spacing = unit(0, "pt"), 
          legend.key.height = unit(8, "pt"), 
          legend.spacing.x = unit(1, "pt"), 
          legend.key.width  = unit(0, "pt"), 
          legend.background = element_blank(), 
          legend.box.background = element_rect(colour = "black"))

ggsave(filename = "desired_look.png", p + desired_display, width = 2, height = 2)
ggsave(filename = "just.png", p + just, width = 2, height = 2)
ggsave(filename = "just_box_probs.png", p + just_box_probs, width = 2, height = 2)

Desired Look:

Legend is too far from the plot, plot itself is squished:

Legend box doesn't move with legend text:

Here is one way to change the dimensions of the plot without the relative dimensions of the legend changing:

library(tidyverse)
data_points = data.frame(x=rnorm(n=16), 
                         y=rnorm(n=16), 
                         col=c(rep("red",8), rep("blue",8)),
                         shap=rep(c(rep("sq",4), rep("tr",4)), 2))
                     
ggplot(data_points, aes(x=x, y=y, color=col, shape=shap)) + 
  geom_point() +
  theme_classic() +
  scale_x_continuous(sec.axis = dup_axis(labels = NULL)) +
  scale_y_continuous(sec.axis = dup_axis(labels = NULL, 
                                         name = NULL,
                                         breaks = NULL)) + 
  theme(legend.spacing = unit(0, "pt"), 
        legend.key.height = unit(8, "pt"), 
        legend.spacing.x = unit(1, "pt"), 
        legend.key.width  = unit(0, "pt"), 
        legend.background = element_blank(), 
        legend.box.background = element_rect(colour = "black"),
        legend.justification = "top",
        legend.margin=margin(4,4,4,4),
        legend.box.spacing = margin(0.5))