Problem: I have videos from which I want to extract the optical flow, even though they do not have clear features to track.
Details:
I have a small channel through which I flow water. I add tracer particles to the water, and image those using a microscope. The idea is that by observing the movement of the tracer particles, I can get the flowrate/direction of the water in the channel.
Unfortunately, my microscope cannot distinguish individual particles, but the optical flow is still recognizable (by humans) - see this movie for instance (raw data below¹), where particles flow from top right to bottom left (without the yt compression it looks more recognizable, I promise).
What would be a way of getting the optical flow without identifying clear features, such as in this movie? Many methods for tracing flows exist², but I am a bit lost in what I can and should use. Also because even dense optical flow algorithms seem to rely on features recognizable in several frames after each other in some way - which is not so easy here.
¹ Two examples of raw data: video 1, video 2.
² Among many others: Trackpy, OpenCV, this entire list
Based on the non-lossy TIFFs, I conclude that there's strong high frequency noise that hides a coarse pattern that moves across the view.
A lowpass would suppress the high frequency noise and allow you to see, and a program to more easily react to, the low frequency content.
Filters for lowpassing:
medianBlur
, most robust to noise/outliersGaussianBlur
: good old gaussian blurblur
: dumb old box blur, if combined with straight decimation it's equivalent to resize with INTER_AREA
stackBlur
, something newish in OpenCV that's fast and almost looks like a gaussianSince you're planning to calculate optical flow, you should also downsample the data, so the calculations have less data to work on, being done quicker. Since there is no high frequency content anyway (after filtering), decimation causes no loss of information.
Downsampling can be done with:
pyrDown()
, which is specifically for image pyramids, but also works well for cheap downsampling by powers of tworesize()
, if(!) you make sure to use INTER_AREA
. All other modes have the potential to cause unbounded aliasing as the scale factor decreases (no real lowpassing in the call)resize()
with nearest neighbor sampling is straight decimation. Acceptable to do if the signal was lowpassed. No reason to do that because combined/fused operations are usually cheaper than individual operations.The first two you can apply without a lowpass because they include a lowpass: pyrDown()
runs a Gaussian before decimating, which is best. INTER_AREA
calculates an arithmetic mean ("box blur") over all source pixels contributing to a destination pixel, which is good but may pass some limited high frequency information (i.e. show some aliasing).