pythonpandaspandas-groupbypandas-apply

Keep NaN groups when using GroupBy apply

I'm looking to keep the structure when using apply on a GroupBy object where some groups are NaN. Using dropna=False does not appear to help, NaN groups are still lost with apply.

mux = pd.MultiIndex.from_arrays([['a', 'a', np.nan, 'b', 'b'], ['t', 'u', np.nan, 'w', 'y']],
                                names=['level1', 'level2'])
df = pd.DataFrame({'col': [0, np.nan, np.nan, 3, 4]}, mux)
df

                col
level1  level2  
a       t       0.0
        u       NaN
NaN     NaN     NaN
b       w       3.0
        y       4.0

df = df.groupby(['level2'], dropna=False).apply(lambda x: x)
df

               col
level1  level2  
a       t      0.0
        u      NaN
b       w      3.0
        y      4.0
Solution

  • Seems like this is more like a bug for call the index without level

    out = df.groupby(level='level2', dropna=False).apply(lambda x: x)
Out[706]: 
               col
level1 level2     
a      t       0.0
       u       NaN
b      w       3.0
       y       4.0
NaN    NaN     NaN