Python constrained optimization on external code

I am trying to wrap an external code in Python and then perform a constrained optimization on it. I think scipy.optimize should work, but I am unsure how to set up the constraints. My actual code cannot be shared but I think I can illustrate the problem without it.

Let's assume my external code is in this form:

def external_code(x):
    # does stuff with the x[0] through x[2] inputs
    # calculates all the outputs, y[0] through y[5]
    return y

Let's suppose my optimization objective is to minimize y[0] which is one of the things calculated in that external code, subject to a number of constraints. First, there are limits on acceptable values of the individual x inputs which I think I can handle like this: bounds = Bounds([0.0, -np.inf, 0.0], [10.0, 10.0, 2.0]). There are also constraints which must be respected for a valid optimum. Unlike the examples in the documentation or lecture examples I found, my constraints are on other outputs of the external code (i.e., y[1] through y[5]), not on a direct/simple combination of the x inputs. I think I need to set up a constraint function something like this:

def constraint_fcn(y):
    return np.array([5.0 - y[1],
                     1.0 - y[2],
                     1.0 - (y[3]+y[4]+y[5])
                    ])

I believe I would need to wrap my external code in an additional layer such that there is only one return value for the optimizer to operate on, i.e.,

def minimize_this(x):
    y = external_code(x)
    return y[0]

The call to the optimizer would then be something like:

x0 = np.array([1.0, 0.0, 1.0])
res = scipy.optimize.minimize(minimize_this, x0, method='SLSQP', bounds=bounds,
                              constraints={'fun': constraint_fcn, 'type': 'ineq'})

The problem is that while minimize_this is set up to operate on x0 and each successive pick of x by the optimizer, constraint_fcn is not meant to work on x, but instead on those additional y values returned by the call to external_code. I could theoretically set up constraint_fcn to instead take in x, make its own call to external_code, and then set the limits based on the y[1] through y[5], but that would then mean that minimize_this and constraint_fcn are making separate calls to external_code which is not acceptable for computational performance reasons. I have seen in the documentation that both the function to be minimized and the constraint function can be passed additional arguments, but I was not seeing how to pass things from the function to me minimized to the constraint functions. I suspect that I just need to reformulate things, but I have been unable to find examples that are more like my setup.

I think you'll be forced to evaluate your external function more. You can update your constraint_fcn to take x as an argument and get the y-values through the external function evaluation.

def constraint_fcn(x):
    y = external_code(x)
    return np.array([5.0 - y[1],
                     1.0 - y[2],
                     1.0 - (y[3]+y[4]+y[5])
                    ])

Edit: You may be able to cache your results using something like the functools.cache decorator.

import functools

@functools.cache
def cached_external(x):
    return external_code(x)

def minimize_this(x):
    return cached_external(x)[0]

def constraint_fcn(x):
    y = cached_external(x)
    return np.array([5.0 - y[1],
                     1.0 - y[2],
                     1.0 - (y[3]+y[4]+y[5])
                    ])

Since you're working with floats, you may need to write your own caching decorator that checks the result match within floating point precision.