pythonnumpyclassadditionnumpy-ndarray

Different behavior in custom class between left addition/right addition with a numpy array


I am writing a class where one of the stored attributes is cast to an integer in the constuctor. I am also overloading left/right addition, where adding/subtracting an integer means to shift this attribute over by that integer. In principle, addition commutes in all contexts with my class, so as a user I would not expect there to be any difference between left and right addition.

However, my code disagrees. I would naively expect addition by a numpy array to fail. The code behaves as expected for left addition, but erroneously runs with right addition! Here is an extemely minimal working example of what I mean:

import numpy as np

class foo:

    def __init__(self, val):
        self.val = int(val)

    def __add__(self, other):
        if isinstance(other, self.__class__):
            return foo(self.val + other.val)
        try:
            return foo(self.val + int(other))
        except:
            raise ValueError(f"unsupported operand type(s) for +: {type(self).__name__} and {type(other).__name__}")

    __radd__ = __add__  

    def __repr__(self):
        return str(self.val)   

Then, when I run the following block:

a = foo(6)
b = np.arange(10)
>>> print(f"left addition by 'a': {a + b}")

I get an exception as expected. But when I run this code block, it runs just fine.

>>> print(f"right addition by 'a': {b + a}")
right addition by 'a': [6 7 8 9 10 11 12 13 14 15] # Adds by b element-wise

It seems as though right addition is defaulting to numpy array's addition overload method, as expected from the documentation for radd (emphasis mine)

These methods are called to implement the binary arithmetic operations (+, -, *, @, /, //, %, divmod(), pow(), **, <<, >>, &, ^, |) with reflected (swapped) operands. These functions are only called if the left operand does not support the corresponding operation [3] and the operands are of different types. [4] For instance, to evaluate the expression x - y, where y is an instance of a class that has an __rsub__() method, type(y).__rsub__(y, x) is called if type(x).__sub__(x, y) returns NotImplemented.

So I think that numpy is cleverly realizing that my own class implements addition with the dtype inside the numpy array and loops over it.

For what it's worth, I don't dislike this functionality at all. I only dislike that my addition operation behaves differently between left and right ones. I'm looking for either an error in both cases (probably preferrable for simplicity) or to work in both cases. I'm not sure what the best way to unify their behavior is.

Some naive ideas that come to mind (such as having the left addition call addition in the reversed order to invoke numpy's interpretation of things) seems like it might lead to some unintuitive behavior.

I had hoped to get a better idea of what was happening by looking at the numpy source code for 'add' directly, but the documentation page for np.add doesn't have a link to its source code the same way others do (such as np.atleast_1d)...

Are there any clean workarounds to this problem?

---

As a side question, the way the addition operation is written above closely mirrors the structure of addition in my actual class, where the form looks kind of like this:

def __add__(self, other):
    if isinstance(other, self.__class__):
        ...
    try:
        ...
    except:
        raise ValueError(...)

My thought process was that if the two objects are the same type, I know how to handle this perfectly since I wrote the code for this class. But, if a user is trying to add some other type, for my purposes, all that really matters is that it has some reasonable interpretation as an integer. So I ended up with some weird mix between 'look before you leap' and 'easier to ask forgiveness than permission' coding styles where I'm not sure if it's sacrilegious or not. Is this considered bad coding style?


Solution

  • Numpy provides some hooks for this.

    In this case you probably want to implement class.__array_ufunc__() on your class. If you simply define it with None it will raise an exception:

    __array_ufunc__ = None 
    

    Alternatively, you can actually implement something:

    def __array_ufunc__(self, ufunc, method, *inputs):
        print(f"{ufunc} called with {inputs}") 
        return self
    
    # b + a just 
    # prints: <ufunc 'add'> called with (array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]), 6)
    # and returns itself