rweb-scrapingmovies

Creating movie scraping script


I'm trying to create a movie scraping script, but after running the last line of code from below I'm met with a

Error in rbind(deparse.level, ...) : 
  numbers of columns of arguments do not match 

And I'm not sure why.

  rottenrate <- function(movie){
              require(RJSONIO)
              link <- paste("http://www.omdbapi.com/?t=", movie, "&y=&plot=short&r=json&tomatoes=true", sep = "")
              jsonData <- fromJSON(link)
              return(jsonData)
            }
            vrottenrate <- Vectorize(rottenrate, "movie", SIMPLIFY = F)

val <- "http://www.fandango.com/valkilmer/filmography/p38142"
    val_movies <- readHTMLTable(val)
    val_movies <- as.data.frame(val_movies)
    val_movie_titles <- subset(val_movies, select = c(NULL.Title))
    val_movie_titles <- as.character(val_movie_titles$NULL.Title) 

val_completed <- do.call(rbind, lapply(vrottenrate(val_movie_titles), function(x) as.data.frame(t(x), stringsAsFactors = FALSE)))

Solution

  • An easily solution is to use bind_rows from dplyr package or rbindlist from data.table package:

     kk<-lapply(vrottenrate(val_movie_titles), function(x) as.data.frame(t(x), stringsAsFactors = FALSE))
    
     library(dplyr)
     bind_rows(kk)
    
    library(data.table)
    rbindlist(kk,fill=TRUE)