pythonnumpyscipylinear-regressionnan

Linear regression of arrays containing NAN values


I have two arrays, say varx and vary. Both contain NaN values at various positions. However, I would like to do a linear regression on both to show how much the two arrays correlate. This was very helpful so far.

However, using the following

slope, intercept, r_value, p_value, std_err = stats.linregress(varx, vary)

results in NaNs for every output variable. What is the most convenient way to take only valid values from both arrays as input to the linear regression? I heard about masking arrays, but am not sure how it works exactly.


Solution

  • You can remove NaNs using a mask:

    mask = ~np.isnan(varx) & ~np.isnan(vary)
    slope, intercept, r_value, p_value, std_err = stats.linregress(varx[mask], vary[mask])