pythongoogle-colaboratoryfinanceyahoo

Can't extract table from yahoo finance

I try to run the below code but still can't get the proper output in Excel with headers. Please help.

!pip install selenium
!apt-get update # to update ubuntu to correctly run apt install
!apt install chromium-chromedriver
!cp /usr/lib/chromium-browser/chromedriver /usr/bin
import sys
sys.path.insert(0,'/usr/lib/chromium-browser/chromedriver')
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
wd = webdriver.Chrome('chromedriver',chrome_options=chrome_options)

lists = ["FBRX", "GNLN", "TISI"]

result=[] 

for list in lists:
  url = "https://finance.yahoo.com/quote/{list}/profile?p={list}"
  wd.get(url.format(list=list))
  add = wd.find_element_by_xpath('//*[@id="Col1-0-Profile-Proxy"]/section/section[1]/table/tbody').text

  print(list,add)
  result.append([list,add])

The intended format will be in the below pic. pic

Solution

  • If an Excel spreadsheet isn't mandatory, then your "result" variable can be saved as a txt file in the same formatting in terms of rows and columns by including the following lines of code afterwards:

    import numpy as np
import pandas as pd
# Mout Drive;
from google.colab import drive
myGoogleDrive = drive.mount('/content/drive', force_remount = True)
%cd "/content/drive/My Drive/Colab Notebooks"
fileName = 'InsertNameForFile' + '.txt'
np.savetxt(fileName, result, delimiter = ", ", fmt = '%s')

    To read that saved txt file at a later point in time:

    savedResultsFile = pd.read_csv(fileName, header = None).copy()