pythonpandasdataframemissing-dataffill

Filling in missing data using "ffill"

I have the following data

4/23/2021   493107
4/26/2021   485117
4/27/2021   485117
4/28/2021   485117
4/29/2021   485117
4/30/2021   485117
5/7/2021    484691

I want it to look like the following:

4/23/2021   493107
4/24/2021    485117
4/25/2021    485117
4/26/2021   485117
4/27/2021   485117
4/28/2021   485117
4/29/2021   485117
4/30/2021   485117
5/1/2021    484691
5/2/2021    484691
5/3/2021    484691
5/4/2021    484691
5/5/2021    484691
5/6/2021    484691
5/7/2021    484691

So it uses date below to fill in the missing data. I tried the following code:

 df['Date']=pd.to_datetime(df['Date'].astype(str), format='%m/%d/%Y')   
 df.set_index(df['Date'], inplace=True)    
 df = df.resample('D').sum().fillna(0)
 df['crude'] = df['crude'].replace({ 0:np.nan})
 df['crude'].fillna(method='ffill', inplace=True)

However, this results in taking the data above and getting the following:

4/23/2021   493107
4/24/2021   493107
4/25/2021   493107
4/26/2021   485117
4/27/2021   485117
4/28/2021   485117
4/29/2021   485117
4/30/2021   485117
5/1/2021    485117
5/2/2021    485117
5/3/2021    485117
5/4/2021    485117
5/5/2021    485117
5/6/2021    485117
5/7/2021    969382

Which does not match what I need the output to be.

Solution

  • Try replace 0 with bfill instead of ffill:

    import pandas as pd

df = pd.DataFrame({
    'crude': {'4/23/2021': 493107, '4/26/2021': 485117,
              '4/27/2021': 485117, '4/28/2021': 485117,
              '4/29/2021': 485117, '4/30/2021': 485117,
              '5/7/2021': 484691}
})
df.index = pd.to_datetime(df.index)

df = df.resample('D').sum()

df['crude'] = df['crude'].replace(0, method='bfill')

print(df)

    df:

                 crude
2021-04-23  493107
2021-04-24  485117
2021-04-25  485117
2021-04-26  485117
2021-04-27  485117
2021-04-28  485117
2021-04-29  485117
2021-04-30  485117
2021-05-01  484691
2021-05-02  484691
2021-05-03  484691
2021-05-04  484691
2021-05-05  484691
2021-05-06  484691
2021-05-07  484691