pythonpandas

Python code to remove records with two or more empty fields

I want to remove records which have 2 or more NaNs, but so far all the code I have found online is either for removing one NaN or not relevant to this situation (e.g. Thresh, any and all).

The code I use for at least 1 NaN is df_exercise.isnull().any(axis=1).

I’m not sure how to adapt this specifically to 2 or more NaNs.

Solution

  • The key to success is to count NaN cases in each row and check whether it is less than your threshold. You can get it running:

    df_exercise.isnull().sum(axis=1) < 2

    And to drop rows exceeding this threshold (keep rows within the threshold), run:

    df_exercise = df_exercise[df_exercise.isnull().sum(axis=1) < 2]