R - tidy augment confidence interval

I am wondering how I can compute confidence interval using the broom package.

What I am trying to do is simple and standard :

set.seed(1)
x <- runif(50)
y <- 2.5 + (3 * x) + rnorm(50, mean = 2.5, sd = 2)
dat <- data.frame(x = x, y = y)
mod <- lm(y ~ x, data = dat)

Using visreg I can plot regression models with CI very simply with :

library(visreg)
visreg(mod, 'x',  overlay=TRUE) 

I am interesting in reproducing this using broom and ggplot2, so far I only achieved this :

 library(broom) 

 dt = lm(y ~ x, data = dat) %>% augment(conf.int = TRUE)  
 ggplot(data = dt, aes(x, y, colour = y)) + 
  geom_point() + geom_line(data = dt, aes(x, .fitted, colour = .fitted)) 

The augment funciton doesn't compute conf.int. Any clue how I can add some smooth confidence invervals ?

 geom_smooth(data=dt, aes(x, y, ymin=lcl, ymax=ucl), size = 1.5, 
        colour = "red", se = TRUE, stat = "smooth")

Using the broom output, you can do something like this:

ggplot(data = dt, aes(x, y)) + 
  geom_ribbon(aes(ymin=.fitted-1.96*.se.fit, ymax=.fitted+1.96*.se.fit), alpha=0.2) +
  geom_point(aes(colour = y)) + 
  geom_line(aes(x, .fitted, colour = .fitted)) +
  theme_bw()

I moved colour=y into geom_point() because you can't apply a colour aesthetic to geom_ribbon.