pythonopencvimage-processingcomputer-visionscikit-image

Python: ski.transform.warp from scikit-image library does not work as expected


I have recently started playing with some image processing and am relatively new to the field. I am at a lost as to why I get unexpected results when using the following code from the python scikit-image library to do a simple projective transform.

# TRANSFORM
imgGray = cv2.cvtColor(cv2.imread(imgPath), cv2.COLOR_BGR2GRAY)

dst = np.array([[1000, 100], 
                [1000, 200], 
                [800, 200], 
                [800, 100]])

src = np.array([[500, 0],
                [500, 200],
                [0, 200],
                [0, 0]])

tform3 = ski.transform.ProjectiveTransform()
tform3.estimate(src, dst)
tf_img_warp = ski.transform.warp(imgGray, tform3, output_shape=(200, 500))


## PLOT
plt.figure(num=None, figsize=(8, 6), dpi=80)
fig, ax = plt.subplots(1,2, figsize=(15, 10), dpi = 80)
ax[0].set_title(f'Original', fontsize = 15)
ax[0].imshow(imgGray)
ax[0].set_axis_off()
ax[1].set_title(f'Transformed', fontsize = 15)
ax[1].imshow(tf_img_warp)
ax[1].set_axis_off()


patch1 = Circle((dst[0][0],dst[0][1]),
                 10, facecolor = color)
patch2 = Circle((dst[1][0],dst[1][1]), 
                 10, facecolor = color)
patch3 = Circle((dst[2][0],dst[2][1]),
                 10, facecolor = color)
patch4 = Circle((dst[3][0],dst[3][1]), 
                 10, facecolor = color)
patch5 = Circle((src[0][0],src[0][1]), 10, 
                 facecolor = color)
patch6 = Circle((src[1][0],src[1][1]), 10, 
                 facecolor = color)
patch7 = Circle((src[2][0],src[2][1]), 10, 
                 facecolor = color)
patch8 = Circle((src[3][0],src[3][1]), 10, 
                 facecolor = color)

ax[0].add_patch(patch1)
ax[0].add_patch(patch2)
ax[0].add_patch(patch3)
ax[0].add_patch(patch4)
ax[1].add_patch(patch5)
ax[1].add_patch(patch6)
ax[1].add_patch(patch7)
ax[1].add_patch(patch8)

By reusing the code above, the results seem good at times e.g.

Results as expected

Other times, the results seem to have change the image in unexpected ways e.g. from the original image below, we can see some black tiles and some white patches on the top left corner of the selected area to be transformed.

Selected area in original image 2

However, the output from the transformation seems to alter this pattern.

Transformed image does not reflect reality

As we can some of the black tiles are gone, together with the white patch on the top left corner. I have attempted multiple times to check for any silly mistakes on my end but I found none that is causing this unexpected result.

The output image has not been cropped, as can be seen from the number of rows and columns of tiles is the same as the selected area in the original image.

Can any image processing sifu out there help this newcomer understand what is going on? I am absolutely dumbfounded. Thanks!

EDIT p/s: I have also tried using opencv's perspective transform. But the results are the same as well.

M = cv2.getPerspectiveTransform(src, dst)
imgWarp = cv2.warpPerspective(img, M, (img.shape[1], img.shape[0]), flags=cv2.INTER_LINEAR)
imgWarp = Image.fromarray(imgWarp)

Solution

  • I eventually found out the problem (some thanks to Christoph Rackwitz for his prompt suggestion).

    Referring to the image below,

    enter image description here

    During the 1st phase, the red points were inputted by the user. The y-axis at this phase was increasing in the positive direction when moving upwards (like shown in the image below).

    enter image description here

    During the 2nd phase, the y-axis was increasing in the positive direction when moving DOWNWARDS (like shown in the first image). So the red points were plotted as the blue points instead, which subsequently caused the warping operation to occur using that portion of the image.

    So to revert back to the originally intended points, I need only invert the y-coordinates i.e.

    src[:, 1] = img.height - src[:, 1]
    

    Hope this helps others when dealing with images in different phases and conditions.