jsonrrjsonio

Simplifying a POSIX node with RJSONIO::fromJSON()

I have the following vector of double values, x, where each element represents a POSIX date-time

x <- c(1417621083, 1417621204, 1417621384, 1417621564, 1417621623)

I am using the RJSONIO package, and would like to continue to do so.

As an exercise, I'd like to convert these values into JSON text and then read them back into R again, but am having trouble getting the date-time representations into a nice simplified list result. In JSON, the dates need to be in a special format so the values in x are converted to the following:

dates <- c("/new Date(1417621083)", "/Date(1417621204)", "/Date(1417621384)", 
           "/Date(1417621564)", "/Date(1417621623)")

When I run dates with a second arbitrary vector through the RJSONIO parser, everything seems to go smoothly.

library(RJSONIO)
make <- toJSON(list(date = dates, value = LETTERS))

Then when I parse the new JSON text using the stringFun option with the R-json C routine for dates, the result is a two-element list, the first element being a list and the second an atomic vector. 

(read <- fromJSON(make, stringFun = "R_json_dateStringOp"))
# $date
# $date[[1]]
# [1] "2014-12-03 07:38:03 PST"
# 
# $date[[2]]
# [1] "2014-12-03 07:40:04 PST"
# 
# $date[[3]]
# [1] "2014-12-03 07:43:04 PST"
# 
# $date[[4]]
# [1] "2014-12-03 07:46:04 PST"
# 
# $date[[5]]
# [1] "2014-12-03 07:47:03 PST"
# 
# 
# $value
# [1] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M"
# [14] "N" "O" "P" "Q" "R" "S" "T" "U" "V" "W" "X" "Y" "Z"

But I was expecting a list of two vectors, and I would rather have it in the form of

# $date
# [1] "2014-12-03 07:38:03 PST" "2014-12-03 07:40:04 PST"
# [3] "2014-12-03 07:43:04 PST" "2014-12-03 07:46:04 PST"
# [5] "2014-12-03 07:47:03 PST"
# 
# $value
# [1] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M" "N" "O" "P" "Q"
# [18] "R" "S" "T" "U" "V" "W" "X" "Y" "Z"

I tried several ways to simplify the result from within the call to fromJSON(), and none of them have worked. Here are a couple of my attempts:

Using a handler : This simplifies the result but fails to reformat the dates

h1 <- basicJSONHandler(simplify = TRUE)
fromJSON(make, handler = h1, stringFun = "R_json_dateStringOp")
# $date
# [1] "/new Date(1417621083)" "/Date(1417621204)"    
# [3] "/Date(1417621384)"     "/Date(1417621564)"    
# [5] "/Date(1417621623)"    
# 
# $value
# [1] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M"
# [14] "N" "O" "P" "Q" "R" "S" "T" "U" "V" "W" "X" "Y" "Z"

Trying the simplify argument : I tried several different varieties of this and none worked.

fromJSON(make, simplify = StrictCharacter)
# $date
# [1] "/new Date(1417621083)" "/Date(1417621204)"    
# [3] "/Date(1417621384)"     "/Date(1417621564)"    
# [5] "/Date(1417621623)"    
#
# $value
#  [1] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M"
# [14] "N" "O" "P" "Q" "R" "S" "T" "U" "V" "W" "X" "Y" "Z"

Is there a way to simplify the result for the dates in the call to fromJSON()?

Solution

  • I think you cannot get the coercion of dates and their simplification to a vector in the same time. For the simple reason that this is not (yet) implemented in RJSONIO. Indeed as you mention the simplification is done using one the flag : StrictLogical, StrictNumeric and StrictCharacter which create logicals, numbers or characters vectors. Maybe you should contact the maintainer to add StrictPosixct flag for POSIXct dates.

    Using stringFun can't help because it receives a scalar element(a character string) and it is not aware of other vector elements. You can check this by defining an R function as stringFun parameter and put a browser within it.

    convertJSONDate <-
  function(x)
  {
     if(grepl('Date',x)){
       val <- sub('.*[(]([0-9]+).*','\\1',x)
       return(structure(as.numeric(val)/1000, class = c("POSIXct", "POSIXt")))
     }
     x
   }

    I guess you want to do the coercion/simplification when you parse your json for a performance reason. I would use a different strategy :

    1. I coerce my numeric values to a POSIXct and I will store them as a character in a well formatted dates. This is better then the special ( ugly) "new Date(.. ,date") RJSONIO date format. Remember that json format is a standard format that can be parsed by other languages ( python, js,..)
    2. Then parse my dates as a normal character and I use the fast fasttime package to coerce it to POSIXct vector.

    here some code to show this:

    ## coerce x to dates a well formatted dates
dd <- as.character(as.POSIXct(x,origin = '1970-01-01' , tz = "UTC"))
## read it again in a fast way
fastPOSIXct(fromJSON(make)$date)

[1] "2014-12-03 16:38:03 CET" "2014-12-03 16:40:04 CET" "2014-12-03 16:43:04 CET" "2014-12-03 16:46:04 CET" "2014-12-03 16:47:03 CET"