Add panel border to ggplot2

I have been asked to place a full border around my plot below:

Using panel.border = element_rect(colour = "black") results in losing in the plot becoming blank.

I can't use theme_bw() as it does not have the same functionality as the usual theme, the code I am currently using is below:

graph<-ggplot(d,aes(x=d$AOE, y=d$MEI)
            )+
  geom_point(shape=20, size=3)+
  geom_rug()+
  annotate("text", x = -1.1, y = 14000, label = "27/04/2011") +
  annotate("text", x = -1.3, y = 10400, label = "03/04/1974") +
  xlab("MEI")+
  ylab("AOE")+
  scale_y_log10()+
  theme(axis.text.y   = element_text(size=14),
        axis.text.x   = element_text(size=14),
        axis.title.y  = element_text(size=14),
        axis.title.x  = element_text(size=14),
        panel.background = element_blank(),
        panel.grid.major = element_blank(), 
        panel.grid.minor = element_blank(),
        axis.line = element_line(colour = "black")
  )

graph 

Any advice on how to get a full black border would be very much appreciated!

To use panel.border you also have to specify a blank fill using fill=NA.

Try this:

library(ggplot2)

ggplot(mtcars, aes(mpg, disp)) + geom_point() + geom_rug() +
  theme(axis.text.y   = element_text(size=14),
        axis.text.x   = element_text(size=14),
        axis.title.y  = element_text(size=14),
        axis.title.x  = element_text(size=14),
        panel.background = element_blank(),
        panel.grid.major = element_blank(), 
        panel.grid.minor = element_blank(),
        axis.line = element_line(colour = "black"),
        panel.border = element_rect(colour = "black", fill=NA, linewidth=5)
  )