pythonpython-3.xmultiprocessing-manager

How to access property attribute from multiprocess Proxy class in Python?

I am trying to get attribute from my Proxy class but I don't quite understand the implementation (as per https://docs.python.org/3.9/library/multiprocessing.html?highlight=multiprocessing#multiprocessing.managers.BaseManager.register)

I understand I need to pass exposed and method_to_typeid to the .register() method because if I don't I only have access to "public" methods and not attributes.

Here is my code;

from multiprocessing import Process
from multiprocessing.managers import BaseManager

class CustomManager(BaseManager):
    # nothing
    pass

class TestClass:

    def __init__(self):
       self._items = []

    @property
    def items(self):
        return self._items

    def fill_items(self):
        self._items.append(1)


if __name__ == "__main__":
    CustomManager.register(
        'TestClass', 
        TestClass,
        exposed=('items', 'fill_items'),
        method_to_typeid={'items': 'list'}
    )
    manager = CustomManager()
    manager.start()
    shared_object = manager.TestClass()
    
    p = Process(target=shared_object.fill_items)
    p.start()
    p.join()
    print(shared_object.items)
    #print(shared_object.items())

I would expect this to return my list but it returns a reference to the method;
Output:

<bound method items of <AutoProxy[TestClass] object, typeid 'TestClass' at 0x7feb38056670>>

But when I try to call it as a method i.e. shared_object.items() I get;

File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py", line 824, in _callmethod
    raise convert_to_error(kind, result)
TypeError: 'list' object is not callable

Which makes sense because it is an attribute that contains list type value not a method. So but then why when I try to call it as an attribute I get it's reference not the value?

Tried following the official documentation and checked already asked questions, for most of which the solution was to add NamespaceProxy, but it looks like now instead of implementing our own NamespaceProxy the correct way is to just pass the two extra args for .register() method.

Solution

  • The solution here is just to use threading instead of multiprocessing. ChatGPT got pretty close to the implementation I needed but could not resolve the issue without changing the implementation of my classes. In the end it makes more sense to use threads anyway because;