rjsondataframe

How Do I Extract JSON component and save it as a data.frame in R?


I am wrestling with some JSON data from eodhd. I only want to change only one part of the json list into a data frame. This is giving me some big headaches:

$  library(jsonlite)
$  dh <- read_json("tickers-gspc.json")$HistoricalTickerComponents
$  str(dh[1:3])
List of 3
 $ 0:List of 6
  ..$ Code       : chr "A"
  ..$ Name       : chr "Agilent Technologies Inc"
  ..$ StartDate  : chr "2000-06-05"
  ..$ EndDate    : NULL
  ..$ IsActiveNow: int 1
  ..$ IsDelisted : int 0
 $ 1:List of 6
  ..$ Code       : chr "AAL"
  ..$ Name       : chr "American Airlines Group"
  ..$ StartDate  : chr "2015-03-23"
  ..$ EndDate    : chr "2024-09-23"
  ..$ IsActiveNow: int 0
  ..$ IsDelisted : int 0
 $ 2:List of 6
  ..$ Code       : chr "AAP"
  ..$ Name       : chr "Advance Auto Parts Inc"
  ..$ StartDate  : chr "2015-07-09"
  ..$ EndDate    : chr "2023-08-25"
  ..$ IsActiveNow: int 0
  ..$ IsDelisted : int 0

My first attempt was do.call("rbind", dh[1:3]), but although R does seem to know that I am now dealing with a data frame of 3 obs and 6 vars, write.csv(...) still doesn't like it.

> xx <- as.data.frame(do.call("rbind", dh[1:3]))
> write.csv(xx, file="abc.csv")
Error in utils::write.table(xx, file = "abc.csv", col.names = NA, sep = ",",  : 
  unimplemented type 'list' in 'EncodeElement'

Must be a simple way to fix this...


Solution

  • As mentioned in the comment:

    xx = as.data.frame(do.call('rbind', dh[1:3]))
    write.csv(xx, file='abc.csv')
    

    issues

    Error in utils::write.table(xx, file = "abc.csv", col.names = NA, sep = ",", : unimplemented type 'list' in 'EncodeElement'

    The error seems to be due to the NULL. If you do not have an option to fix the import, consider writing a replace function r. A basic recursive version

    r = \(l) lapply(l, \(j) if (is.list(j)) r(j) else if (is.null(j)) NA else j)
    
    X = do.call('rbind.data.frame', r(dh[1:3]))
    write.csv(X, file='abc.csv')
    

    Data dh

    dh = list(
      list(
        Code = "A",
        Name = "Agilent Technologies Inc",
        StartDate = "2000-06-05",
        EndDate = NULL,
        IsActiveNow = 1,
        IsDelisted = 0
      ),
      list(
        Code = "AAL",
        Name = "American Airlines Group",
        StartDate = "2015-03-23",
        EndDate = "2024-09-23",
        IsActiveNow = 0,
        IsDelisted = 0
      ),
      list(
        Code = "AAP",
        Name = "Advance Auto Parts Inc",
        StartDate = "2015-07-09",
        EndDate = "2023-08-25",
        IsActiveNow = 0,
        IsDelisted = 0
      )
    )