c++opencvimage-processingimage-segmentationmean-shift

Difference between meanShiftFiltering and meanShiftSegmentation in OpenCV


Could somebody please explain me the difference between gpu::meanshiftFiltering and gpu::meanShiftSegmentation?

The documentation doesn't really help.

Maybe it's a different question, but my goal is to extract the objects' boundaries, if somebody could help me with that, it would be great.


Solution

  • TL;DR

    mean shift filtering is designed to reduce noise and improve the quality of the image, it returns a "cleaner" image.
    On the other hand, mean shift segmentation segments the image into regions that have roughly the same color. It returns a map of each pixel to its corresponding segment.
    If you want object boundaries you should use the mean shift segmentation.


    Please see the seminal work of Comaniciu and Meer Mean shift: A robust approach toward feature space analysis (PAMI 2002) describing the "mean shift" procedure.

    "Mean shift" is a numeric procedure designed to search for modes ("concentration points") of a distribution defined through samples. Put in simpler words in the context of image processing: you can consider all pixels of an image as samples in RGB-XY space. A "red" region in the image can be viewed as a "high probability" of pixels having red color in XY coordinates of this region. Thus you can think of an image as a collection of RGB-XY samples defining a distribution in this space.
    The "mean-shift" procedure is designed to find regions in RGB-XY space with high probability ("mode" = local max points of the distribution).

    Why is this perspective useful? you might ask yourself. Well, consider a noisy image, the noise drives the pixels away from the modes of the original distribution in RGB-XY space. If you can "shift" each pixel from its current (noisy) color towards a mode of the distribution you are likely to reduce the noise.

    Thus, mean shift filtering changes colors of pixels to resemble the most dominant color in their vicinity.

    Taking one step forward, you can ask yourself "which one of the modes each pixel belongs to?". Thus if you have m modes (in RGB-XY space) you can assign each pixel to one of these m modes, effectively segmenting the image into m segments. This procedure is called mean shift segmentation.