pythonpandaschained-assignment

df.loc causes a SettingWithCopyWarning warning message

The following line of my code causes a warning :

import pandas as pd

s = pd.DataFrame(np.random.randint(0,100,size=(100, 4)), columns=list('ABCD'))
s.loc[-1] = [5,np.nan,np.nan,6]
grouped = s.groupby(['A'])
for key_m, group_m in grouped:
    group_m.loc[-1] = [10,np.nan,np.nan,10]

C:\Anaconda3\lib\site-packages\ipykernel\__main__.py:10: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

According to the documentation this is the recommended way of doing, so what is happening ?

Thanks for your help.

Solution

  • The documentation is slightly confusing.

    Your dataframe is a copy of another dataframe. You can verify this by running bool(df.is_copy) You are getting the warning because you are trying to assign to this copy.

    The warning/documentation is telling you how you should have constructed df in the first place. Not how you should assign to it now that it is a copy.

    df = some_other_df[cols]

    will make df a copy of some_other_df. The warning suggests doing this instead

    df = some_other_df.loc[:, [cols]]

    Now that it is done, if you choose to ignore this warning, you could

    df = df.copy()

    or

    df.is_copy = None