pythonpython-3.xpython-multiprocessingmultiprocessing-manager

How to access classes stored in a multiprocessing.Manager list?


I can access and change the attributes of classes in a list like this:

class TestClass:
    def __init__(self):
        self.value = 1

instances = [TestClass()]
instances[0].value = 42
print(instances[0].value)  # 42

However, when using the multiprocessing.Manager, my code seems to be without any effect:

from multiprocessing import Manager
with Manager() as manager:
    instances = manager.list([TestClass()])
    instances[0].value = 42
    print(instances[0].value)  # 1

How to properly store an iterable with instances of a class using the multiprocessing module?


Solution

  • This question is a duplicate of Adding values to set contained in Multiprocessing.Manager().list()

    As @torek stated, the proxy manager can't propagate your changes to the underlying object, so you have to replace it entirely.

    Note Modifications to mutable values or items in dict and list proxies will not be propagated through the manager, because the proxy has no way of knowing when its values or items are modified. To modify such an item, you can re-assign the modified object to the container proxy:

       # create a list proxy and append a mutable object (a dictionary)
       lproxy = manager.list()
       lproxy.append({})
       # now mutate the dictionary
       d = lproxy[0]
       d['a'] = 1
       d['b'] = 2
       # at this point, the changes to d are not yet synced, but by
       # reassigning the dictionary, the proxy is notified of the change
       lproxy[0] = d
    

    So, here it's your solution:

    with Manager() as manager:
        instances = manager.list([TestClass()])
        newInstance = TestClass()
        newInstance.value = 42
        instances[0] = newInstance
        print(instances[0].value)