Rasterizing coordinates for an irregular polygon changes original shape

I am trying to smoothen the edges of an irregular polygon in R, namely to turn its sharp corners into round edges. I am trying to do this using smoothr::smooth, but this function operates on objects from packages sf or sp while all I have is a set of coordinates. Somehow, the result of turning my data.frame into a SpatialPolygonsDataFrame object (an object class from package sp) is a rectangle whose limits are the extreme limits of the original polygon. Does anyone know how to turn my set of coordinates in an object of a class compatible with smoothr::smooth while maintaining the original polygon shape? Here is what I did, partially following instructions at this page:

rm(list=ls()) # my compulsive habit of making sure R's memory is a clean slate

# Example dataset:
dd <- data.frame(
        Lon = c(18.95379, 18.82409, 18.58987, 18.80541, 18.92427, 19.00264),
        Lat = c(-32.42492, -32.32498, -31.89642, -31.73606, -32.16217, -32.37052)
        )

plot(0,0,
    xlim=c(18.5,19.1), ylim=c(-32.5,-31.6), 
    xlab="Longitude", ylab="Latitude"
    )
polygon(dd[,"Lon"],dd[,"Lat"], border="red")

# To make it smooth I plan on using
library(smoothr)
# But smoothr:: smooth works on objects from packages sf or sp so I need to  convert dd.
#convert to spatial points
library(sp)
coordinates(dd) = ~Lon + Lat
# convert to raster
library(raster)
rr <- raster::raster(dd)
#convert raster to polygons
sp = rasterToPolygons(rr, dissolve = T)
map(sp, add=T, col="green", fill=F)
# somehow my irregular polygon turned into a rectangle.
sps <- smooth(sp, method = "ksmooth", smoothness=5)
# this works, but of course is only rounding the corners of sp
map(sps, add=T, col="blue", fill=F)

In red is my original polygon from the data.frame dd, in green is object sp, in blue is the smooth version of sp, sps. Function smooth does the job, the problem is somewhere in the conversion of dd into an sp-compatible object. I suspect the problem is caused by raster() but I am not sure why or of how to fix it. Many thanks in advance.

Here I've used sf because I personally find that much easier:

library(sf)
library(smoothr)

# Example dataset:
dd <- data.frame(
  Lon = c(18.95379, 18.82409, 18.58987, 18.80541, 18.92427, 19.00264),
  Lat = c(-32.42492, -32.32498, -31.89642, -31.73606, -32.16217, -32.37052)
)

# cast to polygon, use multipoint first though.   
polygon <- as.matrix(dd) %>%
  sf::st_multipoint() %>%
  sf::st_cast("POLYGON")

#  smooth polygon
polygon_smoothed <- smoothr::smooth(polygon, method = "ksmooth", smoothness = 0.5)

# plot to check
plot(polygon, col = "red")
plot(polygon_smoothed, col = "blue", add = T)