pythonimage-processingscikit-image

What is the difference between skimage view_as_window and sklearn extract_patch_2d, in the context of extracting array of patches from an image?


I have an array of 2D images; an array of [n x w x h], n being the number of images. I want to extract patches from each 2D image. A 2D image would not be divided equally by the patch size (meaning I should use padding). I do not want overlapping patches. I came across two functions in skimage and sklearn for this: view_as_window() and extract_patches_2d(). My questions are,

  1. Why there are two methods?
  2. Which one suits my use case the most?

Solution

  • Since you want to extract patches from a 2D image, sklearn.feature_extraction.image.extract_patches_2d is the right choice, because it does this and nothing else, and you can expect it to have been optimised for that task.

    Though something like skimage.util.view_as_windows may have a similar outcome for the 2D case you're interested in, it is not the specific purpose it was written for. This function provides a rolling window view of an n-dimensional array, and although in your case 'n' may be 2, it works for any n.

    As a result, skimage.util.view_as_windows is likely not optimised for the use case you have in mind, or may get weighed down with additional logic in the future that would affect your use case only negatively.

    Even if sklearn.feature_extraction.image.extract_patches_2d ended up calling skimage.util.view_as_windows in the background, it would still make sense to use the more specific call, to benefit from potential future optimisation or avoid divergence from your use case.

    As an analog consider a math library that has a function to solve a quadratic equation, and another function that can solve any polynomial equation under certain conditions. If you only need to solve quadratic equations, you'd pick the first one, even though the second one would work as well. But why both exist is obvious, and similar reasoning applies for the case you're asking about.