Hi I am new to Disco and integrating existing code to it. Is Disco capable to call map/reduce function as a function within a class instead of global function? The following code might explain more clearly.
class Segmenter(object):
def map_fun(line, params):
....
def reduce_fun(iter, params):
....
def disco_mp(self):
job = Job().run(input=["raw://word_to_segment_......"],
map=map_fun,
reduce=reduce_fun)
...
The result of execution is
NameError: global name 'map_fun' is not defined
But if I change map_fun, reduce_fun into global function, it would work fine as expected. However I still have to find a way to make it work as class functions, is there any way to do it ?
Thanks,
Chandler
You need static method, you can do this with decorator:
class Segmenter(Job):
map = staticmethod(map_fun)
reduce = staticmethod(reduce_fun)
@staticmethod
def map_fun(line, params):
....
@staticmethod
def reduce_fun(iter, params):
....
def disco_mp(self):
job = self.run(input=["raw://word_to_segment_......"])
Note that you will not have access to self
in both map_fun and reduce_fun, and this is why params
exists. Also note that Job.run
is now self.run
and Segmenter
extends Job
.