I'm trying to apply SA (Simulated Annealing) to the problem of linear regression.
For example: I now have 200 points, I want to fit a line and get k and b of it.
My problem is: I'm trying to solve this problem with MATLAB. But I'm confused about how to pick the next state. I tried this in my program:
next_k = k + (2*rand-1);
next_b = b + (2*rand-1);
I think it's not very good, and the result k and b is not accurate. I want a more effective way to make disturbance to the current state in SA. I think by doing this, I can get a more accurate result in this problem. In this problem, the cost function is actually the same as least-squares method. I'm just want to apply
SA to the estimate problem.
With SA, sometimes fast and straightforward proposals work very well, as they are fast to implement.
(?) Did it work well?
If it doesnt converge the last mile, you might want to try variations where similar proposals are chosen but with smaller randomness, to optimize the last few tiny steps. Large noise will work fast in the beginning, but might not get you close to (local) optimum.
Like the basic example in https://pypi.org/project/frigidum/
It is using 2 proposals, one with large random noise, and one with tiny.
Other variation: Randomly only adjust k or b, that way the annealing will be more smooth.