Is it possible to mix ctypes.Structure with regular python fields?

Just going the straight way like this, doesn't seem to work in python 3.

import ctypes

class cA(ctypes.Structure):
    _fields_ = [("a", ctypes.c_ubyte)]

    def __init__(self):
        super().__init__()
        self.c = 3

z = cA.from_buffer_copy(b'0')
print (z)
print (z.c)
print (dir(z))

Traceback (most recent call last):
  File "polymorph.py", line 12, in <module>
    print (z.c)
AttributeError: 'cA' object has no attribute 'c'

So ctypes.Structure factory classmethod from_buffer_copy() doesn't seem to use the default constructor __init__. Is this correct and intended by ctypes.Structure?

Ok even if not trying to workaround this, by overwriting the class method from_buffer_copy raises another issue. Like adding to class cA:

@classmethod
def from_buffer_copy(cls,buffer, offset=0):
    #obj = super().from_buffer_copy(b'2',offset)
    obj = cls.from_buffer_copy(b'2',offset)
    obj.__init__()
    return obj

Traceback (most recent call last):
  File "polymorph.py", line 17, in <module>
    z = cA.from_buffer_copy(b'0')
  File "polymorph.py", line 13, in from_buffer_copy
    obj = cls.from_buffer_copy(b'2',offset)
  File "polymorph.py", line 13, in from_buffer_copy
    obj = cls.from_buffer_copy(b'2',offset)
  File "polymorph.py", line 13, in from_buffer_copy
    obj = cls.from_buffer_copy(b'2',offset)
  [Previous line repeated 996 more times]
RecursionError: maximum recursion depth exceeded

This creates a RecursionError, because same class method is called again and again, but Structure.from_buffer_copy can't be called either, because it lacks _fields_.

Using super() above doesn't help either. It raises an AttributeError:

Traceback (most recent call last):
  File "polymorph.py", line 17, in <module>
    z = cA.from_buffer_copy(b'0')
  File "polymorph.py", line 12, in from_buffer_copy
    obj = super().from_buffer_copy(b'2',offset)
AttributeError: 'super' object has no attribute 'from_buffer_copy'

This seems to be similar to Using super with a class method Looks like a special Python 3 thing either.

So ctypes.Structure seems to be a real special kind of class. As doing the same with regular Python class is no issue. Like:

import ctypes

class A:
    def __init__(self):
        self.a = "Tex"

    @classmethod
    def factory(cls):
        return cls()

class B(A):
    def __init__(self):
        super().__init__()
        self.b = "bTex"
        
z = B.factory()
print (z)
print (z.b)
print (dir(z))

So ... Is there a way to mix ctypes.Structure with regular Python class fields or better just don't do it and better embed ctypes.Structure inside other regular classes for this just to work around this?

Solution

.from_buffer_copy() doesn't call __init__. But use a normal constructor and it works, although I don't recommend this since a ctypes.Structure is meant to exactly wrap the fields of a C structure and represent its memory layout. How would additional Python fields be represented? Also note that the default constructor of a ctypes.Structure takes arguments to initialize its fields, so they must be passed on to the superclass for this to work:

import ctypes

class cA(ctypes.Structure):
    _fields_ = [("a", ctypes.c_ubyte)]

    def __init__(self, *args):
        print(f'__init__({args=})')
        super().__init__(*args)
        self.c = 3

z = cA(123)
print (z.a, z.c)

Output:

__init__(args=(123,))
123 3

Here's a workaround if you want from_buffer_copy to work as well:

import ctypes as ct
import struct

class Example(ct.Structure):
    _fields_ = (('a', ct.c_int),
                ('b', ct.c_int))

    def __init__(self, *args):
        super().__init__(*args)
        self.c = 55

    @classmethod
    def _from_buffer_copy(cls, buffer, offset=0):
        obj = Example._from_buffer_copy_original(buffer, offset)
        obj.__init__()
        return obj

    def __repr__(self):
        return f'Example(a={self.a}, b={self.b}, c={self.c})'

# Patch from_buffer_copy to eliminate the recursion problem.
Example._from_buffer_copy_original = Example.from_buffer_copy
Example.from_buffer_copy = Example._from_buffer_copy

ex1 = Example(11, 22)
print(ex1)
ex2 = Example.from_buffer_copy(struct.pack('ii', 22, 33))
print(ex2)

Output:

Example(a=11, b=22, c=55)
Example(a=22, b=33, c=55)

Suggested way to handle the problem instead:

import ctypes as ct
import struct

# Strictly a C structure wrapper.
class Example(ct.Structure):
    _fields_ = (('a', ct.c_int),
                ('b', ct.c_int))

    def __repr__(self):
        return f'Example(a={self.a}, b={self.b})'

# Class to contain structure and additional data.
class Info:
    # Creates the structure from a buffer and initialize additional data
    def __init__(self, buffer, c, d):
        self.ex = Example.from_buffer_copy(buffer)
        self.c = c
        self.d = d

    # Debug representation
    def __repr__(self):
        return f'Info(buffer={bytes(self.ex)}, c={self.c}, d={self.d})'

buffer = struct.pack('ii', 11, 22)
info = Info(buffer, 33, 44)
print(f'{info = }\n  {info.ex = }')
# An ideal debug representation can reproduce the original object when evaluated.
info2 = eval(repr(info))
print(f'{info2 = }\n  {info2.ex = }')

Output:

info = Info(buffer=b'\x0b\x00\x00\x00\x16\x00\x00\x00', c=33, d=44)
  info.ex = Example(a=11, b=22)
info2 = Info(buffer=b'\x0b\x00\x00\x00\x16\x00\x00\x00', c=33, d=44)
  info2.ex = Example(a=11, b=22)