pythonpandasdataframeduplicity

Pandas throws weird exception when trying to apply function to duplicate columns

Why am I getting the following error message? I am trying to apply a function to a duplicate column. Please don't tell me that the solution is to do something like df["a"] = 2 * df["a"]; this is a dumbed down example of something more complicated I am working on right now.

>>> df = pd.DataFrame({"a" : [0,1,2], "b" : [1,2,3]})
>>> df[["a", "a"]].apply(lambda x: x[0] + x[1], axis = 1)
Traceback (most recent call last):
  File "C:\Users\Alexander\Anaconda3\lib\site-packages\pandas\indexes\base.py", line 1980, in get_value
    tz=getattr(series.dtype, 'tz', None))
  File "pandas\index.pyx", line 103, in pandas.index.IndexEngine.get_value (pandas\index.c:3332)
  File "pandas\index.pyx", line 111, in pandas.index.IndexEngine.get_value (pandas\index.c:3035)
  File "pandas\index.pyx", line 154, in pandas.index.IndexEngine.get_loc (pandas\index.c:3955)
  File "pandas\index.pyx", line 169, in pandas.index.IndexEngine._get_loc_duplicates (pandas\index.c:4236)
TypeError: unorderable types: str() > int()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Alexander\Anaconda3\lib\site-packages\pandas\core\frame.py", line 4061, in apply
    return self._apply_standard(f, axis, reduce=reduce)
  File "C:\Users\Alexander\Anaconda3\lib\site-packages\pandas\core\frame.py", line 4157, in _apply_standard
    results[i] = func(v)
  File "<stdin>", line 1, in <lambda>
  File "C:\Users\Alexander\Anaconda3\lib\site-packages\pandas\core\series.py", line 583, in __getitem__
    result = self.index.get_value(self, key)
  File "C:\Users\Alexander\Anaconda3\lib\site-packages\pandas\indexes\base.py", line 2000, in get_value
    raise IndexError(key)
IndexError: (0, 'occurred at index 0')
Solution

  • IIUC you need change x[0] and x['1'] to x.a, because there is no columns 0 and 1:

    a = df[["a", "a"]].apply(lambda x: x.a + x.a, axis = 1)
print (a)
   a  a
0  0  0
1  2  2
2  4  4

    But if duplicity columns have different values use iloc:

    import pandas as pd

df = pd.DataFrame({"a" : [0,1,2], "b" : [1,2,3]})
df.columns = ['a','a']
print (df)
   a  a
0  0  1
1  1  2
2  2  3

df['sum'] = df.iloc[:,0] + df.iloc[:,1]
print (df)
   a  a  sum
0  0  1    1
1  1  2    3
2  2  3    5

    What is same as:

    df['sum'] = df.a.apply(lambda x: x.iloc[0] + x.iloc[1], axis = 1)
print (df)
   a  a  sum
0  0  1    1
1  1  2    3
2  2  3    5