pythonnumpynumexpr

Use object attribute in numexpr expression


I am trying to use an object attribute in a numexpr expression. The most obvious way of doing this:

import numpy as np
import numexpr as ne

class MyClass:
    def __init__(self):
        self.a = np.zeros(10)

o = MyClass()

o.a

b = ne.evaluate("o.a+1")

Results in the following error

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-22-dc90c81859f1> in <module>()
     10 o.a
     11 
---> 12 b = ne.evaluate("o.a+1")

~/.local/lib/python3.5/site-packages/numexpr/necompiler.py in evaluate(ex, local_dict, global_dict, out, order, casting, **kwargs)
    799     expr_key = (ex, tuple(sorted(context.items())))
    800     if expr_key not in _names_cache:
--> 801         _names_cache[expr_key] = getExprNames(ex, context)
    802     names, ex_uses_vml = _names_cache[expr_key]
    803     arguments = getArguments(names, local_dict, global_dict)

~/.local/lib/python3.5/site-packages/numexpr/necompiler.py in getExprNames(text, context)
    706 
    707 def getExprNames(text, context):
--> 708     ex = stringToExpression(text, {}, context)
    709     ast = expressionToAST(ex)
    710     input_order = getInputOrder(ast, None)

~/.local/lib/python3.5/site-packages/numexpr/necompiler.py in stringToExpression(s, types, context)
    296         names.update(expressions.functions)
    297         # now build the expression
--> 298         ex = eval(c, names)
    299         if expressions.isConstant(ex):
    300             ex = expressions.ConstantNode(ex, expressions.getKind(ex))

<expr> in <module>()

AttributeError: 'VariableNode' object has no attribute 'a'

Consulting another question, I was able to get a less than satisfactory solution by using numexpr's global_dict:

import numpy as np
import numexpr as ne

class MyClass:
    def __init__(self):
        self.a = np.zeros(10)

o = MyClass()

o.a

b = ne.evaluate("a+1", global_dict={'a':o.a})

That is going to get pretty messy once MyClass has a dozen attributes and there are a few such calls to ne.evaluate.

Is there a simple, clean way of doing this?


Solution

  • Your main concern seems to be the scalability/maintainability of the evaluate call if your object starts having a lot of attributes. You can automate this part by passing vars(o):

    import numpy as np
    import numexpr as ne
    
    class MyClass:
        def __init__(self):
            self.a = np.arange(10000)
            self.b = 2*self.a
    
    o = MyClass()
    
    c = ne.evaluate("a+b", local_dict=vars(o))
    

    Note that I used local_dict because it might be marginally faster to put these names into the local namespace. If there's any chance of the instance attributes clashing with local names in the script (which largely depends on how you name your attributes and what the class does), it's probably safer to pass the vars as the global_dict just like in the question (and for the same reason as noted in a comment).

    You'll still have to keep track of the correspondence between instance attributes and their names in numexpr expressions, but the bulk of the work can be skipped with the above.