python-3.xloss-functionscipy-optimizescipy-optimize-minimize

Early stopping of loss function using scipy.optimize.minimize


I am using scipy.optimize.minimize library to custom write my loss function. https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html

def customised_objective_func(x):
global lag1
global lag2
global lag3
global lag4

pred = []
err = []
df_col_others_copy=df_col_others.copy(deep=True)
df_col_others_copy*=np.array(x)
pred=intercept_val+df_col_others_copy.sum(axis=1)
pred=list(pred)
col_sales=list(df_col_sales)
err = np.array(pred) - np.array(col_sales)
#err = pred - col_sales
obj_func_cust=sum(err**2)/len(err) + (lambda_mult* pen_func(x))

# CONDITION CHECK
avg_lags_val=(lag1+lag2+lag3+lag4)/float(4.0)
perc_change=(np.abs(obj_func_cust-avg_lag_val)/float(avg_lags_val))*100

if perc_change>=2.0:
    break --###?? Sntax for breaking out here??

# retaining last 4 values
curr=obj_func_cust
lag4=lag3
lag3=lag2
lag2=lag1
lag1=curr

if curr=0:
    

return obj_func_cust

myoptions={'maxiter':1000}
results = minimize(customised_objective_func,params,method = "BFGS",options = myoptions) 

I am retaining the values of the loss function calculated for last 4 iterations and I want to check for some variance on them if that condition is met I want to stop the function call (quit further function execution for more iterations even if 1000 iterations are not complete.

How can I achieve this? Would appreciate help here with the keyword to be used/syntx?


Solution

  • Since you need the last four objective function values and you can't terminate the solver inside your objective function, you can write a Wrapper that includes a callback and your objective function:

    class Wrapper:
        def __init__(self, obj_fun, cond_fun, threshold):
            self.obj_fun = obj_fun
            self.cond_fun = cond_fun
            self.threshold = threshold
            self.last_vals = np.zeros(5)
            self.iters = 0
    
        def callback(self, xk):
            if self.iters <= 4:
                return False
            if self.cond_fun(self.last_vals) <= self.threshold:
                return True
    
        def objective(self, x):
            # evaluate the obj_fun, update the last four objective values
            # and return the current objective value
            np.roll(self.last_vals, 1)
            obj_val = self.obj_fun(x)
            self.last_vals[0] = obj_val
            self.iters += 1
            return obj_val
    

    The callback method returns True and terminates the solver if your cond_fun evaluated at the last four values is below a given threshold. Here, xk is just the current point. The objective method is just a wrapper around your obj_fun and updates the last objective values.

    Then, you can use it like this:

    def your_cond_fun(last_obj_vals):
        curr = last_obj_vals[0]
        last4 = last_obj_vals[1:]
        avg_vals = np.sum(last4) / last4.size
        return 100*np.abs(curr - avg_vals) / avg_vals
    
    wrapper = Wrapper(your_obj_fun, your_cond_fun, your_threshold)
    
    minimize(wrapper.objective, params, callback = wrapper.callback, 
             method = "BFGS", options = myoptions)