I am trying to understand Kanade-Lucas-Tomasi tracker. This is the overview (I read from some lectures) of how it should be done:
1. Find harris corners
2. For each corner compute displacement to next frame
using the Lucas-Kanade method
3. Store displacement of each corner, update corner position
4. (optional) Add more corner points every M frames using 1
5. Repeat 2 to 3 (4)
6. Returns long trajectories for each corner point
My doubt in this is do we need to compute the optical flow at some point or just displacement vector is enough to carry out the algorithm? If No then why is optical flow dealt with this topic?
Kanade-Lucas-Tomasi tracker is related to optical flow because the displacement vectors are optical flow vectors but in a sparse sense and not a dense optical flow field. Thats because the tracker is based on the Lucas Kanade method estimating the displacement vector. And Lucas Kanade method is based on the intensity constancy assumption which is solved by a first-order Taylor approximimation this approximation is called optical flow equation and was invented by Horn and Schunk. The Lucas Kanade method is classified as a local optical flow method while most of nowadays most optical flow methods are global methods that produce dense motion fields.