pythonpython-3.xdecoratorpicklepython-decorators

Input/output decorator to pickle function result


Given a function with a parameter a and two other parameters (pickle_from, pickle_to), I'd like to:

With a single function this is straightforward. If pickle_from isn't null, the function just loads the pickled result and returns it. Otherwise, it performs some time-intensive calculation with a, dumps that to pickle_to, and returns the calculation result.

try:
   import cPickle as pickle
except:
   import pickle

def somefunc(a, pickle_from=None, pickle_to=None):

    if pickle_from:
        with open(pickle_from + '.pickle', 'rb') as f
            res = pickle.load(f)

    else:
        # Re-calcualte some time-intensive func call
        res = a ** 2

    if pickle_to:
        # Update pickled data with newly calculated `res`
        with open(pickle_to + '.pickle', 'wb') as f:
            pickle.dump(res, f)

    return res

My question is regarding how to build a decorator so that this process can form a shell around multiple functions similar to somefunc, cutting down on source code in the process.

I'd like to be able to write something like:

@pickle_option
def somefunc(a, pickle_from=None, pickle_to=None)  
    # or do params need to be in the decorator call?
    # remember, "the files are in the computer"
    res = a ** 2
    return res

Is this possible? Something about decorators makes my head explode, so I will politely decline to post here "what I have tried."


Solution

  • This decorator requires a little bit of introspection. Specifically, I've made use of inspect.Signature to extract the pickle_from and pickle_to parameters.

    Other than that, it's a very straightforward decorator: It keeps a reference to the decorated function, and calls it if necessary.

    import inspect
    from functools import wraps
    
    def pickle_option(func):
        sig = inspect.signature(func)
    
        @wraps(func)
        def wrapper(*args, **kwargs):
            # get the value of the pickle_from and pickle_to parameters
            # introspection magic, don't worry about it or read the docs
            bound_args = sig.bind(*args, **kwargs)
            pickle_from = bound_args.arguments.get('pickle_from', \
                                 sig.parameters['pickle_from'].default)
            pickle_to = bound_args.arguments.get('pickle_to', \
                                 sig.parameters['pickle_to'].default)
    
            if pickle_from:
                with open(pickle_from + '.pickle', 'rb') as f:
                    result = pickle.load(f)
            else:
                result = func(*args, **kwargs)
    
            if pickle_to:
                with open(pickle_to + '.pickle', 'wb') as f:
                    pickle.dump(result, f)
    
            return result
    
        return wrapper