How can I add text labels showing the amount of data points in a geom_jitter plot?

I have a geom_jitter plot with a categorical variable in x axis and a continuous variable in y axis. I want to plot on top of the maximum value of each x axis category the total number of data points contained in that category, which would be basically plotting the output of

table(my_df$my_categorical_var)

So far I've tried:

+geom_text(aes(x= my_categorical_var), y= max(my_continuous_var), label=as.vector(my_df$my_categorical_var))

But this outputs an error:

    Error in `geom_text()`:
! Problem while computing aesthetics.
ℹ Error occurred in the 2nd layer.
Caused by error in `check_aesthetics()`:
! Aesthetics must be either length 1 or the same as the data
  (20996)
✖ Fix the following mappings: `label`

Which basically makes you create a geom_text for each label or to modify the dataframe and add a new column for each category containing the number that I want to plot. However there might be something more straigforward.

My desired output would be something like the following plot with a top label with the count of data points of each category:

You could pass a summarized version of the data to your geom_text layer:

library(ggplot2)
library(dplyr)

ggplot(mpg, aes(cyl, hwy)) +
  geom_point(position = position_jitter(width = 1/3)) +
  geom_text(data = . %>% 
              group_by(cyl) %>% 
              summarize(hwy = max(hwy), label = n()),
            aes(label = label), nudge_y = 5, color = 'red')