pythonnumpyopencvcomputer-visionlinear-algebra

Combine several affine transformations get from OpenCV estimateAffinePartial2D


I use OpenCV on Android. I use the goodFeaturesToTrack method to find features in a video frame. Then I take the next video frame and find those features in it using calcOpticalFlowPyrLK and estimate the needed transformation using estimateAffinePartial2D. Up to here everything works as I expect.

What I want to achieve is this, given several transformation matrices from estimateAffinePartial2D I want to have one matrix I can apply to the original frame to get the last frame, i.e. having m1 (frame_0 -> frame_1), m2 ... mn, I want to calculate Mn for a transformation between frame_0 to frame_n.

The matrix returned by estimateAffinePartial2D is 2x3

[[cos(theta)*s, -sin(theta)*s, tx]; 
[sin(theta)*s,  cos(theta)*s, ty]]

I tried adding a row of ones and multiplying two matrices but the result matrix doesn't seem to correspond to a transformation combining the two transformations (the translation getting multiplied seems wrong).

>>> arr1 = np.array([[cos(pi / 4.), -sin(pi / 4.), 100], [sin(pi / 4.), cos(pi / 4.), 100], [1, 1, 1]])
>>> arr2 = np.array([[cos(pi / 2.), -sin(pi / 2.), 50], [sin(pi / 2.), cos(pi / 2.), 50], [1, 1, 1]])
>>> arr_result = np.multiply(arr1, arr2)
>>> print(arr_result)
[[4.32978028e-17 7.07106781e-01 5.00000000e+03]
 [7.07106781e-01 4.32978028e-17 5.00000000e+03]
 [1.00000000e+00 1.00000000e+00 1.00000000e+00]]

Solution

    1. Last row should be [0, 0, 1] instead of [1, 1, 1] in arr1 and arr2
    2. Use numpy.matmul instead of numpy.multiply for matrix multiplication (numpy.multiply is element-wise multiplication, not matrix multiplication)