Detecting mouse movement in a Screen Recording using Computer Vision

I wish to detect and track the mouse cursor in a screen recording video. My goal is to track the mouse cursor in the video and know about the events like closing of a window, exiting an application etc.

Use of normal CV algorithms to track objects does not work as the cursor and background are both dynamic and also there is not a very clear contrast between the two.

Is there a way to tackle this problem?

If the images are uncompressed, all pixel values of the cursor are known exactly and do not change as the cursor is moving (unless there is antialising or transparency on the edges ?).

Then an exact pattern search should do. (Slide the cursor to every possible position and check the matching of the pixels, until all pixels do match.)

If you arrange the search in such a way that the first cursor pixel tried has a different color than the background (the most frequent color), you will maximize the probability of immediate rejection. Even better if the first two pixels tried have different colors. By doing so, the number of pixels tried will get close to twice the image size, which is hard to beat.

Added by Mark Setchell below this line:

It seems cursors do change so you will need to take account of this: