subclassing int in a Cython extension type using new

I want to cythonize code that uses a subclass of "int", which behaves actually like a set (similar to C++ bitset).

The pure python object is instantiated with __new__:

class BitSet(int):
    def __new__(cls, intlist=()):
        val = 0
        for k in intlist:
            val |= (1<<k)
        return super().__new__(cls, val)

However, if I try to make this a Cython type extension like so:

cdef class BitSet(int):
    def __new__(cls, intlist=()):
        # ...

The compilers throws an error:

__new__ method of extension type will change semantics in a future version of Pyrex and Cython. Use __cinit__ instead.

After reading the documentation on __cinit__ and __init__, I don't see exactly how to use them, because __cinit__ should not return a value.

However, I tried the following that compiles:

cdef class BitSet(int):
    cdef BitSet __new__(cls, intlist=()):
        # ...

Using __cinit__ like below also compiles, but I am not sure it is valid, because it looks like we instantiating a base int:

cdef class BitSet(int):
    def __cinit__(self, intlist=()):
        val = 0
        for k in intlist:
            val |= (1<<k)
        self = val

What approach would you use to cythonize such class, and is it actually relevant? For further details, I want my class to have the following methods behave like sets:

__len__: number of bits set to 1 (i.e. number of elements in the set)
__bool__: True if the set is not empty
__iter__: iterate over integer elements
__contains__: test if an integer is in the bitset
__sub__: difference of sets
etc

And I also want to disable some int methods that are not suited, such as __float__, __neg__, __pow__...

Solution

This doesn't really sound like a subclass of int in the sense that it has a substantially different interface to int. It can't (or shouldn't) really be used in place of an int. It sounds more like it's implemented in term of int. So perhaps you should do that instead?

You're right that __cinit__ doesn't do what you want. Your version with self = val definitely won't work - it'll just reassign the self reference within the function.

In Cython 3. you're kind of stuck on using a cdef class anyway because:

TypeError: inheritance from PyVarObject types like 'int' not currently supported

In Cython 0.29.x you get away with it because it isn't checked. However, it'll fall apart if you add anything to the type (e.g. a cdef member, a non-static cdef method). It's so easy to break it that I really don't recommend it. Essentially anything you add to the class will overlay and corrupt the data of the underlying int.

If you insisted on using a cdef class is probably a staticmethod constructor function:

cdef class BitSet(int)
    @staticmethod
    def make_bitset(intlist):
        val = 0
        for k in intlist:
            val |= (1<<k)
        return Bitset(val)

That doesn't prevent people from just calling the constructor themselves of course.

You could obviously use a non-cdef class and that would work.

In summary, this isn't really "correct" use of inheritance, what you want to do isn't really supported by cdef classes, and using a cdef class gives you huge opportunities to corrupt data and break things because older versions of Cython didn't have a check to stop you from doing it.

subclassing int in a Cython extension type using __new__

subclassing int in a Cython extension type using new