Labeling a single point with ggrepel

I am trying to use geom_label_repel to add labels to a couple of data points on a plot. In this case, they happen to be outliers on box plots. I've got most of the code working, I can label the outlier, but for some reason I am getting multiple labels (equal to my sample size for the entire data set) mapped to that point. I'd like just one label for this outlier.

Example:

Here is my data:

dput(sus_dev_data)
structure(list(time_point = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L), .Label = c("3", "8", "12"), class = "factor"), 
    days_to_pupation = c(135L, 142L, 143L, 155L, 149L, 159L, 
    153L, 171L, 9L, 67L, 53L, 49L, 72L, 67L, 55L, 64L, 60L, 122L, 
    53L, 51L, 49L, 53L, 50L, 56L, 44L, 47L, 60L)), row.names = c(1L, 
2L, 3L, 4L, 5L, 6L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 16L, 
17L, 18L, 20L, 21L, 22L, 23L, 24L, 26L, 27L, 28L, 29L, 30L), class = "data.frame")

and my code...

####################################################################################################
# Time to pupation statistical analysis
####################################################################################################

## linear model
pupation_Model=lm(sus_dev_data$days_to_pupation~sus_dev_data$time_point)
pupationANOVA=aov(pupation_Model)
summary(pupationANOVA)

# Tukey test to study each pair of treatment :
pupationTUKEY <- TukeyHSD(x=pupationANOVA, which = 'sus_dev_data$time_point', 
                          conf.level=0.95)

## Function to generate significance labels on box plot
generate_label_df <- function(pupationTUKEY, variable){

  # Extract labels and factor levels from Tukey post-hoc 
  Tukey.levels <- pupationTUKEY[[variable]][,4]
  Tukey.labels <- data.frame(multcompLetters(Tukey.levels, reversed = TRUE)['Letters'])

  #I need to put the labels in the same order as in the boxplot :
  Tukey.labels$treatment=rownames(Tukey.labels)
  Tukey.labels=Tukey.labels[order(Tukey.labels$treatment) , ]
  return(Tukey.labels)
}

#generate labels using function
labels<-generate_label_df(pupationTUKEY , "sus_dev_data$time_point")

#rename columns for merging
names(labels)<-c('Letters','time_point')

# obtain letter position for y axis using means
pupationyvalue<-aggregate(.~time_point, data=sus_dev_data, max)

#merge dataframes
pupationfinal<-merge(labels,pupationyvalue) 

####################################################################################################
# Time to pupation plot
####################################################################################################

# Plot of data
(pupation_plot <- ggplot(sus_dev_data, aes(time_point, days_to_pupation)) +
  Alex_Theme +
  geom_boxplot(fill = "grey80", outlier.size = 0.75) +
  geom_text(data = pupationfinal, aes(x = time_point, y = days_to_pupation, 
                                      label = Letters),vjust=-2,hjust=.5, size = 4) +
  #ggtitle(expression(atop("Days to pupation"))) +
  labs(y = 'Days to pupation', x = 'Weeks post-hatch') +
  scale_y_continuous(limits = c(0, 200)) +
  scale_x_discrete(labels=c("3" = "13", "8" = "18",
                              "12" = "22")) +
    geom_label_repel(aes(x = 1, y = 9), 
                     label = '1')
)

Here's a shorter example to demonstrate what is going on. Essentially, your labels are beng recycled to be the same length as the data.

df = data.frame(x=1:5, y=1:5)

ggplot(df, aes(x,y, color=x)) +
  geom_point() +
  geom_label_repel(aes(x = 1, y = 1), label = '1')

You can override this by providing new data for the ggrepel

ggplot(df, aes(x,y, color=x)) +
  geom_point() +
  geom_label_repel(data = data.frame(x=1, y=1), label = '1')