pythonpython-3.xmultithreadingconcurrent-processing

Why is multiprocessing module not producing the desired result?


import multiprocessing as mp
import os

def cube(num):
    print(os.getpid())
    print("Cube is {}".format(num*num*num))

def square(num):
    print(os.getpid())
    print("Square is {}".format(num*num))

if __name__ == "__main__":
    p1 = mp.Process(target = cube, args = (3,))
    p2 = mp.Process(target = square, args = (4,))
    p1.start()
    p2.start()
    p1.join()
    p2.join()
    print("Done")

I was using the multiprocessing module, but I am not able to print any output from a function using that.

I even tried flushing the stdout using the sys module.


Solution

  • Q : "Why is multiprocessing module not producing the desired result?"

    Why?

    Because it crashes.

    The MWE/MCVE-representation of the problem has a wrong code. It crashes & it has nothing to do with the sys.stdout.flush() :

    >>> cube(  4 )
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 2, in cube
    NameError: global name 'os' is not defined
    

    Solution :

    >>> import os   # be it in the __main__ or in the def()-ed functions...
    
    >>> cube(  4 )
    14165
    Cube is 64
    

    and your mp.Process()-based replicas of the python-process instances will stop crashing too.


    MCVE that works :

    (base) Fri May 29 14:29:33 $ conda activate py3
    (py3) Fri May 29 14:34:55 $ python StackOverflow_mp.py
    This is ____6745::__main__
    This is ____6746::PID
    This is ____6747::PID
    Cube(__3) is _______27.
    Square(__4) is _______16.
    Done.
    Works.
    Q.E.D.
    

    import multiprocessing as mp
    import os
    import sys
    import time
    
    def cube( num ):
        print( "This is {0:_>8d}::PID".format( os.getpid() ) )
        print( "Cube({0:_>3d}) is {1:_>9d}.".format( num, num*num*num ) )
        sys.stdout.flush()
    
    def square( num ):
        print( "This is {0:_>8d}::PID".format( os.getpid() ) )
        print( "Square({0:_>3d}) is {1:_>9d}.".format( num, num*num ) )
        sys.stdout.flush()
    
    if __name__ == "__main__":
        print( "This is {0:_>8d}::__main__".format( os.getpid() ) )
        p1 = mp.Process( target = cube,   args = (3, ) )
        p2 = mp.Process( target = square, args = (4, ) )
        p1.start()
        p2.start()
        p1.join()
        p2.join()
    
        time.sleep( 1 )
    
        print( "Done.\nWorks.\nQ.E.D." )
    

    I copied and pasted your exact code. But I still didn't get the output from the called functions using the multiprocessing libraries
    ā€“ Kartikeya Agarwal 47 mins ago

    So,
    - I opened a new Terminal process,
    - I copied the conda activate py3 command and
    - I hit Enter to let it run, so as to make python3 ecosystem go live.
    - I re-launched the proof-of-solution again python StackOverflow_mp.py and
    - I hit Enter to let it run
    - I saw it working the very same way as it worked last time.
    - I doubt the problem is on the provided twice (re)-validated proof-of-solution side, is it?
    Q.E.D.

    (py3) Fri May 29 19:53:58 $ python StackOverflow_mp.py
    This is ___27202::__main__
    This is ___27203::PID
    Cube(__3) is _______27.
    This is ___27204::PID
    Square(__4) is _______16.
    Done