We are using Google ceres solver for solving optimization problems arising in a computer vision application. We are using an AutoDiffCostFunction to evaluate the residuals, the dense QR solver is able to optimize the function and find a decent minimum. But I'm wondering if using a non-smooth function like fmax, fmin, and relu (or an if/else inside residuals) is considered a bad practice.
Our residuals contains some non-smooth operations :
// compute the area of intersection rectangle
T interArea = fmax(T(0.0), xB - xA) * fmax(T(0.0), yB - yA);
Another snippet that is non-differentiable at zero :
// Generalized Intersection over Union
T g_iou = (a_c > T(0.0)) ? (iou - (a_c - u) / a_c) : T(0.0);
We could replace fmax for example with a smooth-approximation :
(1) fmax(x, y; k) = log( exp(kx) + exp(ky) ) / k
The question is should we systematiccally remove any non-smoothness is residuals even if dense QR is working as intended ?
The choice of linear solver has nothing to do with the smoothness of the objective function. If it works, you are fine, but a smooth objective is what Ceres Solver assume theoretically. The non-smoothness is mild or not terribly widely present, then you are likely okay. That said, people (including me) regularly have conditionals of the sort you are using in our objective functions.