python-3.xweb-scraping

scraping stock futures data from nse website using python


Here is my code

url_oc = "https://www.nseindia.com/get-quotes"
url = f"https://www.nseindia.com/get-quotes/derivatives?symbol=WIPRO"
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) 
             AppleWebKit/537.36 (KHTML, '
                     'like Gecko) '
                     'Chrome/80.0.3987.149 Safari/537.36',
       'accept-language': 'en,gu;q=0.9,hi;q=0.8', 'accept-encoding': 
          'gzip, deflate, br'}
session = requests.Session()
request = session.get(url_oc, headers=headers, timeout=5)
cookies = dict(request.cookies)
response = session.get(url, headers=headers, timeout=5, cookies=cookies).json()

enter image description here

**i am not able to get data

getting error**

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Solution

  • Here is the complete example to grab data from api whatever you desired.

    import requests
    import pandas as pd
    import json
    data=[]
    
    headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36',}
    with requests.session() as req:
        req.get('https://www.nseindia.com/get-quotes/derivatives?symbol=WIPRO',headers = headers)
    
        api_req=req.get('https://www.nseindia.com/api/quote-derivative?symbol=WIPRO',headers = headers).json()
        for item in api_req['stocks']:
            data.append([
                item['metadata']['instrumentType'],
                item['metadata']['openPrice']])
    
    
    cols=['instrumentType','openPrice']
    
    df = pd.DataFrame(data, columns=cols)
    print(df)
    #df.to_csv('info.csv',index = False)
    

    Output:

         instrumentType   openPrice
    0    Stock Futures     646.15
    1    Stock Options      12.05
    2    Stock Options      23.00
    3    Stock Options      29.00
    4    Stock Options      20.40
    ..             ...        ...
    170  Stock Options       0.00
    171  Stock Options       0.00
    172  Stock Options       0.00
    173  Stock Options       0.00
    174  Stock Options       0.00
    
    [175 rows x 2 columns]