pythoncookiespython-requestsyahoo-finance

Yahoo Finance V7 API now requiring cookies? (Python)

url = 'https://query2.finance.yahoo.com/v7/finance/quote?symbols=TSLA&fields=regularMarketPreviousClose&region=US&lang=en-US'
headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36'
}
data = requests.get(url,headers=headers)
prepost_data = data.json()

It seems recently, Yahoo Finance changed their V7 API to require cookies for every request. Running the code above, I get the Invalid Crumb error

{"finance":{"result":null,"error":{"code":"Unauthorized","description":"Invalid Crumb"}}}

This issue seems to also be known in this Github repo: https://github.com/joshuaulrich/quantmod/issues/382

They seem to have a patch that works: https://github.com/joshuaulrich/quantmod/pull/383/commits

But the code is all written in R... Anyone know how to translate this to Python?

Solution

  • This worked for me.

    1. Make HTTP GET call to URL https://fc.yahoo.com. Although this call result in a 404 error, we just need it to extract set-cookie from response headers which is then used in the subsequent calls
    2. Now make an HTTP GET call to the URL https://query2.finance.yahoo.com/v1/test/getcrumb, by including the obtained cookie from the previous response headers. This call will retrieve the crumb value.
    3. Replace [crumb-value] in the following URL and make a HTTP GET call with cookie https://query2.finance.yahoo.com/v7/finance/quote?symbols=TSLA&crumb=[crumb-value]
    4. Cache the cookie value and crumb value to skip first two steps going forward