pythonnumpygenfromtxt

How to use genfromtxt with with brackets in format?


I have a csv file with the following values:

# number,array1,array2
0,[1,2,3,4,5],[6,7,8,9,10]

Now I would like to load these two arrays, but when i run:

new_array = np.genfromtxt(fname='file_name.csv',
           skip_header=1,
           defaultfmt=['%i','%i','%i','%i','%i','%i','%i','%i','%i','%i','%i'],
           deletechars='[,]',
           usecols = (1,2,3,4,5),
           dtype=(int),
           delimiter=',',
           comments='# ',)

Then i get an array with values:

[-1  2  3  4 -1]

Instead of:

[1  2  3  4 5]

If I understand correctly, the problem are the brackets, but I expected that

deletechars='[,]'

would do the trick. How do I get genfromtxt to read these values correctly?


Solution

  • I think deletchars only affects the column names, rather than their data. I think you need a "converter" to remove the square brackets:

    conv = lambda x: int(re.sub(b"[\[\]]", b"", x))
    

    Then you can use:

    In [84]: a = np.genfromtxt(fname='file.csv',
        ...:            skip_header=1,
        ...:            defaultfmt=['%i','%i','%i','%i','%i','%i','%i','%i','%i','%i
        ...: ','%i'],
        ...:            usecols = (1,2,3,4,5),
        ...:            dtype=int,
        ...:            delimiter=',',
        ...:            comments='# ',
        ...:            converters={1:conv,2:conv,3:conv,4:conv,5:conv})
    
    In [85]: a
    Out[85]: array([1, 2, 3, 4, 5])