I'm trying to split a generator into multiple parts so I can do some parallel computing. There is the main generator is produced by g() and gslice() is supposed to produce a subset of it and yield it for the rest of the program to use :
>>> from itertools import product
>>> from string import digits
>>> def g():
... for i in product(digits, repeat=2):
... yield "".join(['aa']+list(i))
>>> def gslice():
... yield itertools.islice(g(), 3,4)
>>> a=gslice()
>>> next(a)
<itertools.islice object at 0x7fc4dc34ad60>
>>> next(next(a))
'aa03'
>>> next(next(a))
Traceback (most recent call last):
File "<console>", line 1, in <module>
Unfortunately as you can see itertools.islice, instead of splitting g(), is yielding some kind of nested object itertools.islice of which you can take the first element once with next(next(a)) but if you try again it produces an error. list() applied to gslice doesn't list anything but return the same object.
The goal of this post is basically to ask why gslice doesn't simply yield a generator that is a subset of g and also to ask how to do that. Imaging if I change repeat=2 in the product method to repeat=10, why certainly wouldn't want to handle this other than through a generator would we.
islice() is itself an iterator (like a generator is an iterator). If you wanted to yield the results of the slice, you can use yield from:
def gslice():
yield from itertools.islice(g(), 3, 4)
yield from requires Python 3.3 or newer. You could also just return the slice:
def gslice():
return itertools.islice(g(), 3, 4)