python-3.xpandasstocknse

Python Error - IndexError: single positional indexer is out-of-bounds


I'm trying to write a Python code to download NSE stocks data for all the listed companies. There are just over 1700 listed companies in NSE India. I want data for only last 120 days. I have writen the below code but its throwing the below error, tried several ways to fix this but its still not working. Any guidance to resolve this issue would be much appreciated. Note that I have declared all the necessary variables that are appearing in my code.

Error: IndexError: single positional indexer is out-of-bounds

My Python version - 3.13

def getHistoricalData(symInfo):
    res = None
    try:
        parseInstrument = urllib.parse.quote(symInfo.instrument_key)
        fromDate = (datetime.now(TIME_ZONE) - timedelta(days=120)) . strftime("%Y-%m-%d")
        todate = datetime.now(TIME_ZONE).strftime("%Y-%m-%d")        
        url = f'https://api.upstox.com/v2/historical-candle/{parseInstrument}/day/{todate}/{fromDate}'        
        res = requests.get(url, headers={'accept': 'application/json', }, params={}, timeout=5.0)
        candleRes = res.json()

        if 'data' in candleRes and 'candles' in candleRes['data'] and candleRes['data']['candles']:
            candleData = pd.DataFrame(candleRes['data']['candles'])
            candleData.columns = ['date', 'open', 'high', 'low', 'close', 'vol', 'oi']
            candleData = candleData[['date', 'open', 'high', 'low', 'close', 'vol', 'oi']]
            candleData['date'] = pd.to_datetime(candleData['date']).dt.tz_convert('Asia/Kolkata')
            candleData['symbol'] = symInfo.tradingsymbol            
            return candleData
        else:
            print('No data', symInfo.instrument_key, candleRes)
    except:
        print(f'Error in data fetch for (symInfo.instrument_key) {res} {e}')

candledfList = []
for i in symboldf.index[:-1]:
    candledfList = []
    res = getHistoricalData(symboldf.iloc[i])
    if res is not None: candledfList.append(res)
    finalDataDf = pd.concat(candledfList, ignore_index=True)

    isCsv = True
    for symData in candledfList[:-1]:
        try:
            filename = symData.iloc[0]['symbol']
            if isCsv:
                filename = f'{filename}.csv'
                symData.to_csv(folder + filename, index=False)
                del candledfList
            else:
                filename = f'{filename}.parquet'
                symData.to_parquet(folder_parquet + filename, engine='pyarrow')
                del candledfList
        except Exception as e:
            print(f'Error {e}')

Solution

  • I have managed to resolve this issue. The reason for the error is very simple -

    1. I had to remove 'i' from "res = getHistoricalData(symboldf.iloc[i])" to make it "res = getHistoricalData(symboldf.loc[i])". This will solve the issue.

      candledfList = [] for i in symboldf.index[:-1]: candledfList = [] res = getHistoricalData(symboldf.iloc[i]) if res is not None: candledfList.append(res) finalDataDf = pd.concat(candledfList, ignore_index=True)

    2. The for loop below is to generate single/separate file for each company stock data. We can remove this and move the logic to for loop above this and generate one single CSV file to capture all companies data. But the problem with this approach is we will lose data for several companies, not sure why. I lost 36 companies data when tried to capture all companies data into one single CSV file.

    isCsv = True for symData in candledfList[:-1]: try: filename = symData.iloc[0]['symbol'] if isCsv: filename = f'{filename}.csv' symData.to_csv(folder + filename, index=False) del candledfList else: filename = f'{filename}.parquet' symData.to_parquet(folder_parquet + filename, engine='pyarrow') del candledfList except Exception as e: print(f'Error {e}')