pythonarraysnumpynan

Removing nan values from a Python List


Can someone see why this is not working? I am trying to remove nan values from my python List/array.

import math
import numpy as np

def clean_List_nan(List):
    Myarray=np.array(List)
    x = float('nan')
    for elem in Myarray:
        if math.isnan(x):
            x = 0.0
    return Myarray


oldlist =[nan, 19523.3211203121, 19738.4276377355, 19654.8478302742, 119.636737571360, 19712.4329437810, nan, 20052.3645613346, 19846.4815936009, 20041.8676619438, 19921.8126944154, nan, 20030.5073635719]

print(clean_List_nan(oldlist))

Solution

  • The control flow in your function makes no sense - you set a variable x to be nan, and then check if it is indeed nan in your loop and set it to 0. You never touch nor check any of the elements of the array.

    To properly convert your nan values to 0, you could simply use numpy.nan_to_num as it appears you're working with NumPy arrays.

    Demo

    In[37]: arr
    Out[37]: 
    array([            nan,  19523.32112031,  19738.42763774,  19654.84783027,
              119.63673757,  19712.43294378,             nan,  20052.36456133,
            19846.4815936 ,  20041.86766194,  19921.81269442,             nan,
            20030.50736357])
    
    In[38]: np.nan_to_num(arr)
    Out[38]: 
    array([     0.        ,  19523.32112031,  19738.42763774,  19654.84783027,
              119.63673757,  19712.43294378,      0.        ,  20052.36456133,
            19846.4815936 ,  20041.86766194,  19921.81269442,      0.        ,
            20030.50736357])
    

    If you're more interested in having a functioning version of an approach for a regular Python list, you might try something like this, or a list comprehension as fafl has provided.

    In[39]: list(map(lambda x: 0.0 if math.isnan(x) else x, oldlist))
    Out[39]: 
    [0.0,
     19523.3211203121,
     19738.4276377355,
     19654.8478302742,
     119.63673757136,
     19712.432943781,
     0.0,
     20052.3645613346,
     19846.4815936009,
     20041.8676619438,
     19921.8126944154,
     0.0,
     20030.5073635719]