pythonpandas

NaN values in Pandas are not being filled by the interpolate function when it's applied to a full dataframe


So, i'm a beginner at the Pandas Python and noticed the interpolate function is pretty interesting, but i have one problem when using the line:

result = df.interpolate(method='linear')

I found out that even though it did filled a lot of the NaN's in 'df', the four first NaN's of Ambient_temp and the first NaN on the Intake_temp column are not filled the way i wanted. Any hints on how to get this working? The interpolation worked very well with every other column besides those two.

Image of said dataframe

Example:

amb_temp = [np.nan, np.nan, 32, 32]
in_temp = [ 29, 27, 23, 22]
volts = [np.nan, 13, 11, 11]

dict = {'ambient_temperature': amb_temp, 'temperature_inside': in_temp, 'volts': volts} 

df = pd.DataFrame(dict)

(it's not exactly the same dataframe, but encapsulates the same problem. I got this one based off and example on 'geeksforgeeks' and used numpy.nan to simulate the absence of data.)


Solution

  • This is it:

    import numpy as np
    import pandas as  pd
    amb_temp = [np.nan, np.nan, 32, 32]
    in_temp = [ 29, 27, 23, 22]
    volts = [np.nan, 13, 11, 11]
    
    dict1 = {'ambient_temperature': amb_temp, 'temperature_inside': in_temp, 'volts': volts} 
    keys = list(dict1.keys())
    
    for k in keys:
        data_array = np.array(dict1[k])
        print("1   {}".format(data_array))
        not_nan = ~np.isnan(data_array)
        indices = np.arange(len(data_array))
        dict1[k] = np.interp(indices, indices[not_nan], data_array[not_nan])
    print(dict1)