I have a dataframe whose values are lists which contains 'nan'. Is there an easy and pythonic way to remove those 'nan' values from lists within the dataframe? I have defined a function which returns a list without 'nan', but how can I apply it to dataframe inplace?
def remove_nan(input_list):
temp_list = [x for x in input_list if x!='nan']
return temp_list
test = ['nan', 'nan', 'SHM System', 'nan', 'nan', 'nan']
remove_nan(test)
['SHM System']
This function works on individual list and returns clean list as shown in the output above. How can I apply this function, or if there is a better way, to remove all 'nan' values from lists within a dataframe? I have tried applymap and apply but didn`t work.
df_combined.applymap(remove_nan)
The following line of code worked for me. Thanks to @piRSquared.
df.applymap(lambda x: [*filter(pd.notna, x)])