pythonoptimizationsubclasspypy

Do PyPy int optimizations apply to classes subclassed from int?


From what I recall, PyPy has special optimizations on built-in types like ints and lists, which is great because they are very common, and it would be wasteful to treat an int like any other object.

If I subclass int specifically, for example

class bitset(int):
    def in_bit(self, i):
        return bool(self & (1 << i))

are these optimizations lost? I do not want to pre-maturely optimize, but I also don't want to be wasteful since I can easily define a function like in_bit(b, i) instead where b is a plain old int.


Solution

  • You'd need to check the generated assembly to be sure, but as far as I can tell by measuring, the answer is no: the basic optimizations are not lost. You also don't need to add __slots__ = (); PyPy doesn't use that for optimizations. The place where it makes a difference is if you make a lot of long-lived instances of your subclass (they take one extra word of memory each, but only if they are long-lived, otherwise the allocation is skipped anyway). There are also a few specific optimizations that would be lost, for example a list containing only (real) ints is stored more compactly. I can't give a complete list; in general, the PyPy interpreter makes various optimizations assuming that real int objects are much more common than subclasses of int, but as it turns out, the later JIT in PyPy can recover some of the missing optimizations when generating assembler.

    EDIT: That said, from a Python language point of view, it's unclear what you gain by sticking utility functions as methods on the subclass of int. You'd need to write bitset(b).in_bit(i) at many places, because I suspect that at many places you don't really know for sure that the integer you're getting is really of the class bitset instead of int. If that's what you have in mind anyway, then sure, bitset(b).in_bit(i) is as well optimized as just calling a plain function with two integer arguments.