How do __slots__ really work and how can you implement your own?

I've been fooling around with __slots__ and searching about them a little, but I'm still confused about some specifics:

I'm aware that __slots__ generates some kind of descriptors:

>>> class C:
...     __slots__ = ('x',)
...

>>> C.x
<member 'x' of 'C' objects>

>>> C.x.__get__
<method-wrapper '__get__' of member_descriptor object at 0x7f001de183a8>

>>> C.x.__get__
<method-wrapper '__get__' of member_descriptor object at 0x7f001de183a8>

But I was wondering: where are the values actually stored?

Because the common recipe/idiom I've seen so far for descriptors is:

>>> class Descr:
...
...     def __init__(self, attrname):
...         self.attrname = attrname
...
...     def __get__(self, obj, cls):
...         return obj.__dict__[self.attrname]
...
...     def __set__(self, obj, val):
...         obj.__dict__[self.attrname] = val
...
... class C:
...
...     def __init__(self, x):
...         self.x = x

When used in addition with __slots__, there are two problems:

1. Clash with names:

>>> class C:
...
...     __slots__ = ('x',)
...
...     def __init__(self, x):
...         self.x = x
...
...     x = Descr('x')
...
Traceback (most recent call last)
 ...
ValueError: 'x' in __slots__ conflicts with class variable

So a workaround is to name the actual attribute '_x'.

2. No __dict__ (unless you explicitly add it to __slots__):

>>> class C:
...
...     __slots__ = ('_x',)
...
...     def __init__(self, x):
...         self._x = x
...
...     x = Descr('_x')
...

>>> c = C(0)

>>> c.x
Traceback (most recent call last)
 ...
AttributeError: 'C' object has no attribute '__dict__'

So you have to use getattr() and setattr() instead.

You can end with a generic descriptor which can work with __dict__ and __slots__:

class WorksWithDictAndSlotsDescriptor:

    def __init__(self, attr_name):
        self.attr_name = attr_name

    def __get__(self, instance, owner):
        try:
            return instance.__dict__[self.attr_name]
        except AttributeError:
            return getattr(instance, self.attr_name)

    def __set__(self, instance, value):
        try:
            instance.__dict__[self.attr_name] = value
        except AttributeError:
            setattr(instance, self.attr_name, value)

(Which won't work as excepted if there is both __slots__ and __dict__.)

But recently I've found a way to hijack the __get__ and __set__ method with a wrapper:

def slot_wrapper(cls, slotname, slot, descriptor):
    '''Wrapper replacing a slot descriptor with another one'''

    class InnerDescr(descriptor):

        def __get__(self, obj, cls):
            print("Hijacking __get__ method of a member-descriptor")
            return slot.__get__(obj, cls)

        def __set__(self, obj, val):
            print("Hijacking __set__ method of a member-descriptor")
            slot.__set__(obj, val)

    return InnerDescr(slotname, cls)

(The use case is to add type checking and data validation, plus enforcing encapsulation.)

So after the creation of a class (or before using a metaclass) you can keep the same name for the slot and your descriptor.

Works well, but feels a little dirty... I think it may be better to implement my own slots to keep using one name for the descriptor. But I don't know how.

So here are some questions:

1. Where are the values actually stored (since there is no dict)? I was thinking it's something implemented in C and not directly accessible with Python code.

2. How can I implement a pure python equivalent without losing performance optimization?

3. Is it preferable to stick with my wrapper?

Where are actually storerd the values (since there is no dict) ? I was thinking it's something implemented in C and not directly accessible with Python code.

Memory is allocated for the PyObject * pointers directly in the object itself. You can see the handling in Objects/typeobject.c. The generated descriptors will access the memory reserved for their slot in an object of the appropriate type.

How can I implement a pure python equivalent without losing performance optimization?

You cannot. The closest you can get is something like extending tuple.

It is prefarable to stick with my wrapper?

No. Don't name your slots the same thing as an attribute you want to be handled by some other descriptor. Doing so is like naming two non-slot descriptors the same thing; you're expressing two contradictory intentions for how you want to handle the attribute with that name.