pythondisco

Cannot call method unless its global from a class


Hi I am new to Disco and integrating existing code to it. Is Disco capable to call map/reduce function as a function within a class instead of global function? The following code might explain more clearly.

class Segmenter(object):
    def map_fun(line, params):
        ....
    def reduce_fun(iter, params):
        ....
    def disco_mp(self):
        job = Job().run(input=["raw://word_to_segment_......"],
                        map=map_fun,
                        reduce=reduce_fun)
        ...

The result of execution is

NameError: global name 'map_fun' is not defined

But if I change map_fun, reduce_fun into global function, it would work fine as expected. However I still have to find a way to make it work as class functions, is there any way to do it ?

Thanks,

Chandler


Solution

  • You need static method, you can do this with decorator:

    class Segmenter(Job):
        map = staticmethod(map_fun)
        reduce = staticmethod(reduce_fun)
    
        @staticmethod
        def map_fun(line, params):
            ....
    
        @staticmethod
        def reduce_fun(iter, params):
            ....
    
        def disco_mp(self):
            job = self.run(input=["raw://word_to_segment_......"])
    

    Note that you will not have access to self in both map_fun and reduce_fun, and this is why params exists. Also note that Job.run is now self.run and Segmenter extends Job.