pythonpython-2.7pandasnatsort

Naturally sorting pandas data rasies error


I have a pandas data from with the following indices

print(df.index)
MultiIndex(levels=[[u'Day 3', u'Day 4', u'Day 5', u'Day 7', u'Day 9'], [u'D1', u'D10', u'D11', u'D12', u'D2', u'D3', u'D4', u'D5', u'D6', u'D7', u'D8', u'D9'], [1.0, 2.0, 3.0]],
       labels=[[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4], [0, 0, 0, 4, 4, 4, 5, 5, 5, 6, 6, 6, 7, 7, 7, 8, 8, 8, 9, 9, 9, 10, 10, 10, 11, 11, 11, 1, 1, 1, 2, 2, 2, 3, 3, 3, 0, 0, 0, 4, 4, 4, 5, 5, 5, 6, 6, 6, 7, 7, 7, 8, 8, 8, 9, 9, 9, 10, 10, 10, 11, 11, 11, 1, 1, 1, 2, 2, 2, 3, 3, 3, 0, 0, 0, 4, 4, 4, 5, 5, 5, 6, 6, 6, 7, 7, 7, 8, 8, 8, 9, 9, 9, 10, 10, 10, 11, 11, 11, 1, 1, 1, 2, 2, 2, 3, 3, 3, 0, 0, 0, 4, 4, 4, 4, 5, 5, 5, 6, 6, 6, 7, 7, 7, 8, 8, 8, 9, 9, 9, 10, 10, 10, 11, 11, 11, 1, 1, 1, 2, 2, 2, 3, 3, 3, 0, 0, 0, 4, 4, 4, 5, 5, 5, 6, 6, 6, 7, 7, 7, 8, 8, 8, 9, 9, 9, 10, 10, 10, 11, 11, 11, 1, 1, 1, 2, 2, 2, 3, 3, 3], [1, 2, 0, 1, 2, 0, 0, 2, 1, 0, 1, 2, 2, 0, 1, 0, 2, 1, 1, 2, 0, 1, 0, 2, 2, 0, 1, 0, 1, 2, 2, 1, 0, 1, 2, 0, 0, 2, 1, 0, 2, 1, 2, 0, 1, 0, 2, 1, 1, 0, 2, 0, 1, 2, 0, 2, 1, 2, 0, 1, 0, 2, 1, 0, 2, 1, 2, 0, 1, 0, 2, 1, 2, 1, 0, 0, 2, 1, 1, 2, 0, 0, 2, 1, 0, 1, 2, 0, 1, 2, 2, 1, 0, 1, 0, 2, 1, 0, 2, 0, 1, 2, 2, 0, 1, 1, 0, 2, 1, 2, 0, 1, 1, 2, 0, 2, 1, 0, 1, 2, 0, 0, 1, 2, 0, 1, 2, 2, 1, 0, 1, 0, 2, 2, 0, 1, 0, 1, 2, 0, 2, 1, 2, 0, 1, 1, 2, 0, 0, 2, 1, 0, 2, 1, 0, 2, 1, 2, 1, 0, 0, 2, 1, 2, 0, 1, 2, 0, 1, 2, 1, 0, 1, 2, 0, 2, 1, 0, 1, 2, 0]],
       names=[u'Interval', u'Device', u'Well'])

I am sorting with the following

 df = df.reindex(index=natsorted(df.index))

With this particular df, however, it returns the follow error.

raise Exception("cannot handle a non-unique multi-index!")
Exception: cannot handle a non-unique multi-index!

Any help would be greatly appreciated.


Solution

  • I made a minimal example and could reproduce your error. It seems it happens, because of the same levels tuple Day 3, D1 and 1.0 in arrays. If you remove one of them it works fine.

    import pandas as pd
    import numpy as np
    from natsort import natsorted
    
    arrays = [[u'Day 3', u'Day 3', u'Day 4', u'Day 5', u'Day 7', u'Day 9', u'Day 3', u'Day 4', u'Day 5', u'Day 7', u'Day 9'],
              [u'D1',    u'D1',    u'D10',   u'D11',   u'D12',   u'D2',    u'D3',    u'D4',    u'D5',    u'D6',    u'D7'],
              [1.0,      1.0,      2.0,      3.0,      1.0,      2.0,      1.0,      2.0,      3.0,      1.0,      2.0]]
    tuples = list(zip(*arrays))
    index = pd.MultiIndex.from_tuples(tuples, names=[u'Interval', u'Device', u'Well'])
    df = pd.Series(np.random.randn(len(arrays[0])), index=index)
    
    print df.index
    
    df = df.reindex(index=natsorted(df.index))
    

    As you mentioned you use several excel files, this may be helpful: Merging multiple dataframes with non unique indexes