pythonpandasfillmissing-data

Forward fill nans and zeros


Suppose I have a dataframe, df1, that has zeros and nans:

dates = pd.date_range('20170101',periods=20)
df1 = pd.DataFrame(np.random.randint(10,size=(20,3)),index=dates,columns=['foo','bar','see'])
df1.iloc[3:12,0] = np.nan
df1.iloc[6:17,1] = 0

What's the succinct way to forward fill both zeors and nans? I tried the below:

df1 = (df1.fillna(method='ffill', inplace=True)).replace(to_replace=0, method='ffill')

AttributeError: 'NoneType' object has no attribute 'replace'

Solution

  • Let's use replace to replace zeros with nan then ffill:

    df1.replace(0, np.nan).ffill()
    

    Output:

                foo  bar  see
    2017-01-01  2.0  1.0    4
    2017-01-02  2.0  2.0    6
    2017-01-03  2.0  8.0    3
    2017-01-04  2.0  6.0    1
    2017-01-05  2.0  8.0    4
    2017-01-06  2.0  9.0    6
    2017-01-07  2.0  9.0    8
    2017-01-08  2.0  9.0    5
    2017-01-09  2.0  9.0    8
    2017-01-10  2.0  9.0    7
    2017-01-11  2.0  9.0    3
    2017-01-12  2.0  9.0    6
    2017-01-13  5.0  9.0    4
    2017-01-14  6.0  9.0    9
    2017-01-15  7.0  9.0    4
    2017-01-16  6.0  9.0    2
    2017-01-17  2.0  9.0    5
    2017-01-18  3.0  1.0    1
    2017-01-19  3.0  8.0    1
    2017-01-20  2.0  5.0    7