python-3.xobjectcythonrefcounting

Cython: casting C struct to pythons object increases refcount


I'm trying to cast a C pointer struct to object type ,while doing so , first field of the struct is being incremented by 1 regardless of what type the field is. Is this behavior expected and I shouldn't cast C pointers to objects or there is a problem with Cython?

example code:

C struct:

struct attr {
   int a;
};

Cython: pxd:

cdef class Attr(object):
   cdef attr * t

pyx:

cdef class Attr(object):
      __init__(self):
         self.t = malloc(sizeof(attr))
         self.t.a = 0

   attr(self):
       return <object>self.t

when creating Attr and running attr method 'a' field of t struct is being incremented (this example is only example of what I'm trying to do not runnable) Thanks for your help


Solution

  • Yes this is expected.

    What you're probably aiming for is that Cython has the ability to autoconvert between C structs and Python dictionaries. This happens without needing to <> cast in your Cython code. For it to work you need to have told Cython about the struct members. You do need to dereference the pointer though: return self.t[0].

    What a cast to <object> does is tells Cython: "this pointer can be directly interpreted as PyObject*, and we are now responsible for reference counting it." The pattern across the C API is that you have various different structs all declared as starting with PyObject_HEAD. You can then cast them to and from PyObject* since the memory layout at the start of the struct is identical. Since your struct does not have this pattern you get nonsense results.


    (Addendum) I think it's worth adding: this has the potential to be much more of a disaster than it currently seems. If what Python thinks is the "refcount" reaches 0 then Python will attempt to deallocate the object. The second part of a PyObject is a pointer to the PyTypeObject that defines the type. A whole load of things (e.g. deallocation, printing the object, just about any interaction with it from Python) can cause Python to try to look up this type object and since the structure doesn't contain a pointer to a valid PyTypeObject then this will end up failing dramatically.