I'm currently trying to calibrate Microsoft Azure Kinect using OpenCV's hand-eye calibration function (cv2.calibrateHandEye()). However, when I plug in 3x1 rotation vectors in as input instead of 3x3 rotation matrices I get totally different results with all methods. Below, I'm sharing the output I get for both cases.
I'm wondering why this discrepancy occurs since I'm using cv2.Rodrigues to convert in between vectors and matrices.
3x1 Rotation Vector case:
(19, 3) (19, 3) (19, 3) (19, 3)
--------------------------------------
Method 0
Rotation:
[[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
Translation:
[[0.]
[0.]
[0.]]
--------------------------------------
--------------------------------------
Method 1
Rotation:
[[nan nan nan]
[nan nan nan]
[nan nan nan]]
Translation:
[[0.]
[0.]
[0.]]
--------------------------------------
--------------------------------------
Method 2
Rotation:
[[-1. 0. 0.]
[ 0. -1. 0.]
[ 0. 0. 1.]]
Translation:
[[0.]
[0.]
[0.]]
--------------------------------------
--------------------------------------
Method 3
Rotation:
[[-0.03388477 0.81429681 0.57945882]
[ 0.37003643 -0.52836555 0.76413538]
Translation:
[[0.]
[0.]
[0.]]
--------------------------------------
--------------------------------------
Method 4
Rotation:
[[nan nan nan]
[nan nan nan]
[nan nan nan]]
Translation:
[[nan]
[nan]
[nan]]
--------------------------------------
3x3 Rotation Matrix case:
(19, 3, 3) (19, 3) (19, 3, 3) (19, 3)
--------------------------------------
Method 0
Rotation:
[[ 0.19681749 0.15272093 -0.96847261]
[-0.9802481 0.01110192 -0.19745987]
[-0.01940435 0.988207 0.15188945]]
Translation:
[[426.01991564]
[ 6.31112392]
[212.62483639]]
--------------------------------------
--------------------------------------
Method 1
Rotation:
[[ 4.38898532e-04 7.05825236e-02 9.97505847e-01]
[-9.99993453e-01 -3.55184483e-03 6.91318082e-04]
[-3.59178096e-03 9.97499620e-01 -7.05805026e-02]]
Translation:
[[ 113.83854629]
[ -64.48053741]
[-155.89394605]]
--------------------------------------
--------------------------------------
Method 2
Rotation:
[[-1.63313542e-04 1.12303599e-01 -9.93673928e-01]
[-9.99994011e-01 -3.45353893e-03 -2.25961739e-04]
[-3.45706791e-03 9.93667940e-01 1.12303491e-01]]
Translation:
[[429.63163945]
[-72.98653944]
[228.31050502]]
--------------------------------------
--------------------------------------
Method 3
Rotation:
[[ 0.27525174 0.87240146 0.403921 ]
[ 0.20267468 0.35804992 -0.91144019]
[-0.93976564 0.33274006 -0.07825982]]
Translation:
[[-126.85624931]
[ 205.9095892 ]
[ 90.09963159]]
--------------------------------------
--------------------------------------
Method 4
Rotation:
[[-0.20970054 0.33923475 -0.91703079]
[-0.94475058 -0.31195916 0.1006371 ]
[-0.25193655 0.88746902 0.38591023]]
Translation:
[[ 716.47999662]
[-112.67040497]
[ 604.15236449]]
--------------------------------------
Probably too late for you, but maybe interesting for someone else: I stumbled upon the same bug and in my case I had a vector of Mat for R_gripper2base. I used push_back to put the actual rotation at the end of the list. When I had vectors as an input everything worked fine, but with the rodrigues matrices push_back did not just put the actual matrix at the end, but changed the value of every other entry in the list aswell. So at the end i had a vector of matrices that were all the same. That's why calibrateHandEye had terrible results.