ggvis: Joining of points with respect to factor levels

I want to connect the values of one variable with respect to the values of other variable. Here in my example I want to connect the values of Y with respect to values of X and Factor. Specifically Y=20 for X=1 and Factor=A should connect with Y=9 for X=2 and Factor=A rather than with Y=15 for X=1 and Factor=A and so.

library(ggvis)
df <- data.frame(X=rep(1:4, times=2), Y=c(20, 9, 15, 5, 25, 18, 29, 10), Factor = rep(LETTERS[1:2], each=4))
df
ggvis(
        data=df
      , x= ~Y
      , y= ~X
      , fill= ~Factor
      , stroke = ~Factor) %>% 
  #group_by(Factor) %>%
  layer_points(shape=~Factor) %>% 
  layer_lines(fillOpacity=0)  %>%
  scale_numeric('y', reverse=TRUE)

If I understand what you want to do correctly, it is just a matter of sorting your data in the right order.

library(dplyr)
library(ggvis)
ggvis(data=df, x= ~Y, y= ~X, fill= ~Factor, stroke = ~Factor) %>% 
  arrange(X) %>%
  group_by(Factor) %>%
  layer_points(shape=~Factor) %>% 
  layer_paths(fill := NA)  %>%
  scale_numeric('y', reverse=TRUE)