pythongevent

Use gevent with parameter in working routine and return result-value?


i have the following working code:

from gevent import monkey; monkey.patch_all()
from gevent.pool import Pool
import time

def work(x):
  time.sleep(1)  # Or use requests.get() or something else IO-related here.
  newVal = x ** 2
  print(x, newVal)

start = time.perf_counter()
p = Pool()
l = [x for x in range(10)]
for i,x in enumerate(l):
  p.spawn(work, x)
p.join()
print(round(time.perf_counter() - start, 3))  # Prints 1.001 or 1.002 for me.

This works fine and is finished within a little bit more than 1 second:

(test) C:\DEV\Python-Diverses\gevent>python try.py
0 0
1 1
2 4
3 9
4 16
5 25
6 36
7 49
8 64
9 81
1.036

Now i want to return the result from the work-function and output all values at the - so i change the code to:

from gevent import monkey; monkey.patch_all()
from gevent.pool import Pool
import time

def work(x):
  time.sleep(1)  # Or use requests.get() or something else IO-related here.
  newVal = x ** 2
  return (newVal)

start = time.perf_counter()
p = Pool()
l = [x for x in range(10)]
ergList = []
for i,x in enumerate(l):
  erg = p.spawn(work, x)
  ergList.append(erg)

p.join()
print(ergList)
print(round(time.perf_counter() - start, 3))  # Prints 1.001 or 1.002 for me.

The whole process still took only 1 second - but the output is not the calculated value - its a Greenlet-instance:

(test) C:\DEV\Python-Diverses\gevent>python try.py
0 0
1 1
2 4
3 9
4 16
5 25
6 36
7 49
8 64
9 81
1.036

(test) C:\DEV\Python-Diverses\gevent>python try.py
[<Greenlet at 0x180704a55e0: _run>, <Greenlet at 0x18070533f40: _run>, <Greenlet at 0x18070563180: _run>, <Greenlet at 0x180705632c0: _run>, <Greenlet at 0x18070563400: _run>, <Greenlet at 0x18070563540: _run>, <Greenlet at 0x18070563680: _run>, <Greenlet at 0x180705637c0: _run>, <Greenlet at 0x18070563900: _run>, <Greenlet at 0x18070563a40: _run>]
1.07

How can i parallelize the process and also get the result-value back and not the Greenlet-instance?


Solution

  • You are misusing the spawn call. When you write:

    p.spawn(work(x))
    

    You are calling work(x) right there. You're not passing the function to p.spawn; you're passing the return value of the function to p.spawn.

    You need to write:

    p.spawn(work, x)
    

    Which produces:

    0
    1
    2
    3
    4
    5
    6
    7
    8
    9
    1.002