computer-visionopticalflow

How to Extract Optical Flow from Videos with Indistinguishable Features?


Problem: I have videos from which I want to extract the optical flow, even though they do not have clear features to track.

Details:

I have a small channel through which I flow water. I add tracer particles to the water, and image those using a microscope. The idea is that by observing the movement of the tracer particles, I can get the flowrate/direction of the water in the channel.

Unfortunately, my microscope cannot distinguish individual particles, but the optical flow is still recognizable (by humans) - see this movie for instance (raw data below¹), where particles flow from top right to bottom left (without the yt compression it looks more recognizable, I promise).

What would be a way of getting the optical flow without identifying clear features, such as in this movie? Many methods for tracing flows exist², but I am a bit lost in what I can and should use. Also because even dense optical flow algorithms seem to rely on features recognizable in several frames after each other in some way - which is not so easy here.

¹ Two examples of raw data: video 1, video 2.

² Among many others: Trackpy, OpenCV, this entire list


Solution

  • Based on the non-lossy TIFFs, I conclude that there's strong high frequency noise that hides a coarse pattern that moves across the view.

    A lowpass would suppress the high frequency noise and allow you to see, and a program to more easily react to, the low frequency content.

    Filters for lowpassing:

    Since you're planning to calculate optical flow, you should also downsample the data, so the calculations have less data to work on, being done quicker. Since there is no high frequency content anyway (after filtering), decimation causes no loss of information.

    Downsampling can be done with:

    The first two you can apply without a lowpass because they include a lowpass: pyrDown() runs a Gaussian before decimating, which is best. INTER_AREA calculates an arithmetic mean ("box blur") over all source pixels contributing to a destination pixel, which is good but may pass some limited high frequency information (i.e. show some aliasing).