ggplot2 Delete elements from legend

I have this plot:

that was made based on this data:

   days variable    value        sd
1    X1  Control 75.03424  3.857730
2    X2  Control 70.17851  2.913420
3    X3  Control 65.01627  9.188947
4    X4  Control 65.70995 10.882072
5    X5  Control 56.98791  8.070014
6    X6  Control 56.64376  4.827183
7    X1   Stress 75.63113  3.207749
8    X2   Stress 70.56030  5.626266
9    X3   Stress 61.56402  7.078610
10   X4   Stress 48.04541 15.287234
11   X5   Stress 43.54458  8.148382
12   X6   Stress 37.51121  9.494008

With this code:

significance <- data.frame(days=c("X4","X5","X6"),value=c(82,70,67), variable=NA)

# Plot
library(ggplot2)
library(extrafont)
library(scales)
library(Cairo)

ggplot(my_mean, aes(x=days, y=value, fill=variable)) +
  geom_bar(stat='identity', position='dodge', width = 0.75) +
  geom_errorbar(aes(ymin = value-sd, ymax = value+sd),
                position = position_dodge(0.75),
                width = 0.3) +
  labs(x='\nDAT',y='μg/cm2\n') +
  scale_y_continuous(limits = c(0,90), expand = c(0,0),
                     breaks = seq(from=0,to=90,by=10)) +
  scale_x_discrete(labels = c(0,7,14,21,27,35)) +
  ggtitle('Chlorophyll Content\n') +
  geom_point(data=significance, aes(y=value),shape='*',size=6) +
  scale_color_manual(values = c("Control" = 'gray45', "Stress" = 'gray')) +
  scale_fill_manual(values = c("Control" = 'gray45', "Stress" = 'gray')) +
  theme(panel.border = element_rect(colour = "black", fill=NA, size=0.5),
        panel.background = element_rect(fill = 'white'),
        plot.title = element_text(hjust = 0.5,family = 'Calibri',face='bold'),
        axis.title = element_text(family = 'Calibri',face = 'bold'),
        axis.text = element_text(family = 'Calibri'),
        legend.text = element_text(family = 'Calibri',face = 'bold'),
        legend.position = c(0.92, 0.91),
        legend.key = element_rect(fill = NA,color = NA),
        legend.title = element_blank(),
        legend.background = element_blank()
        )

Note how I made a dataframe to map my significance points in the plot. But for some reason, these points also were added to the legend, and don't want this to happens. I suspect this could be fixed with some legend function inside the theme configurations, but I wasn't able to find which function to use and what arguments does it would have.

Any suggestion? Or maybe a better approach to map my significance "stars"?

If you don't want the stars to appear in the legend, try geom_point(..., show.legend = F). In fact, show.legend = F is an option in many ggplot2 layers.