google-financepandas-datareader

How come pandas_datareader for google doesn't work?


I tried to grab data from google finance by the following code:

import pandas_datareader.data as wb
import datetime as dt
start = dt.datetime(2015, 1, 1)
end = dt.datetime(2017, 1, 1)

dt = wb.DataReader('FB', 'google', start, end)
dt.head()

and I got this.

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 24697: invalid start byte

But if I changed 'google' to 'yahoo' (by using yahoo finance instead), it works fine. So what's wrong with it?


Solution

  • There is an open issue here.

    A quick fix is below, porting from the source, paring it down and making a few slight tweaks.

    I believe the issue is with the body returned by requests.get() and reading of the resulting bytes. (The traceback agrees with this.) For instance, try data = requests.get(url).content (gets bytes); this will fail. Below, data = requests.get(url).text works.

    I really haven't tested this rigorously but the Google API does appear to be working okay. For instance, the export link generated by url does work just fine at the moment.

    import datetime
    import requests
    from io import StringIO
    from pandas.io.common import urlencode
    import pandas as pd
    
    BASE = 'http://finance.google.com/finance/historical'
    
    
    def get_params(symbol, start, end):
        params = {
            'q': symbol,
            'startdate': start.strftime('%Y/%m/%d'),
            'enddate': end.strftime('%Y/%m/%d'),
            'output': "csv"
        }
        return params
    
    
    def build_url(symbol, start, end):
        params = get_params(symbol, start, end)
        return BASE + '?' + urlencode(params)
    
    
    start = datetime.datetime(2010, 1, 1)
    end = datetime.datetime.today()
    sym = 'SPY'
    url = build_url(sym, start, end)
    
    data = requests.get(url).text
    data = pd.read_csv(StringIO(data), index_col='Date', parse_dates=True)
    
    print(data.head())
    #               Open    High     Low   Close     Volume
    # Date
    # 2017-11-30  263.76  266.05  263.67  265.01  127894389
    # 2017-11-29  263.02  263.63  262.20  262.71   77512102
    # 2017-11-28  260.76  262.90  260.66  262.87   98971719
    # 2017-11-27  260.41  260.75  260.00  260.23   52274922
    # 2017-11-24  260.32  260.48  260.16  260.36   27856514
    

    Edit: The issue should be fixed on version 0.6.0 of pandas_datareader. If not, please reopen it as bashtage requested.