pythonloggingconcurrencymultiprocessing

Log output of multiprocessing.Process


Is there a way to log the stdout output from a given Process when using the multiprocessing.Process class in python?


Solution

  • The easiest way might be to just override sys.stdout. Slightly modifying an example from the multiprocessing manual:

    from multiprocessing import Process
    import os
    import sys
    
    def info(title):
        print title
        print 'module name:', __name__
        print 'parent process:', os.getppid()
        print 'process id:', os.getpid()
    
    def f(name):
        sys.stdout = open(str(os.getpid()) + ".out", "w")
        info('function f')
        print 'hello', name
    
    if __name__ == '__main__':
        p = Process(target=f, args=('bob',))
        p.start()
        q = Process(target=f, args=('fred',))
        q.start()
        p.join()
        q.join()
    

    And running it:

    $ ls
    m.py
    $ python m.py
    $ ls
    27493.out  27494.out  m.py
    $ cat 27493.out 
    function f
    module name: __main__
    parent process: 27492
    process id: 27493
    hello bob
    $ cat 27494.out 
    function f
    module name: __main__
    parent process: 27492
    process id: 27494
    hello fred