rweb-scrapinghtml-tablervestyahoo-finance

The function html_table() in R doesn't give me the full table


I want to web-scrape historical stock market data from the last five years. But my code only gives me a tibble of 100 rows, but the table on the website is much longer. Do you know where I'm wrong?

library("rvest")
library("dplyr")

  url <- "https://finance.yahoo.com/quote/BAC/history?period1=1540512000&period2=1698278400& interval=1d&filter=history&frequency=1d&includeAdjustedClose=true"
  dt <- read_html(url)
  dt %>% html_table(fill = T) 

Solution

  • You can use tidyquant to fetch security prices from Yahoo

    library(tidyverse)
    library(tidyquant)
    
    tq_get(x = "BAC") %>% 
      arrange(desc(date))
    
    # A tibble: 2,725 × 8
       symbol date        open  high   low close   volume adjusted
       <chr>  <date>     <dbl> <dbl> <dbl> <dbl>    <dbl>    <dbl>
     1 BAC    2023-10-27  26.1  26.1  25.0  25.2 64614700     25.2
     2 BAC    2023-10-26  25.5  26.4  25.4  26.1 60921000     26.1
     3 BAC    2023-10-25  25.4  25.6  25.2  25.5 45522400     25.5
     4 BAC    2023-10-24  25.7  25.9  25.4  25.5 55975700     25.5
     5 BAC    2023-10-23  26.1  26.2  25.5  25.6 59857200     25.6
     6 BAC    2023-10-20  26.8  26.9  26.2  26.3 62029800     26.3
     7 BAC    2023-10-19  27.2  27.8  26.9  27.0 58611000     27.0
     8 BAC    2023-10-18  27.5  28.0  27.2  27.3 68371100     27.3
     9 BAC    2023-10-17  27.0  27.9  26.7  27.6 95344200     27.6
    10 BAC    2023-10-16  27.2  27.2  26.8  27.0 56817500     27.0
    # ℹ 2,715 more rows
    # ℹ Use `print(n = ...)` to see more rows
    

    enter image description here