I have a function which is supposed to return the rotation vector and translation vector (rvec and tvec) given some 3d points, some 2d points, and an intrinsics matrix
def solvePnP(points_3d: list[list[float]], points_2d: list[list[int]], camera_mat: np.ndarray, dist_coeffs: np.ndarray):
points_3d = np.array(points_3d)
points_2d = np.array(points_2d)
success, rvec, tvec = cv2.solvePnP(points_3d, points_2d, camera_mat, dist_coeffs)
if success:
return rvec, tvec
When I'm calling the cv2.solvePnP() function, the arguments are
points_3d: 12x3 numpy array = array([[ 0.28758, -0.05071, 0.0015006],
[ 0.27922, -0.0074935, -0.02097],
[ 0.18719, -0.009217, -0.019443],
[ 0.18634, 0.13371, -0.019021],
[ 0.27455, 0.13196, -0.018894],
[ 0.27863, 0.18347, 0.0046991],
[ -0.12992, 0.17744, 0.014193],
[ -0.11417, 0.1198, -0.01239],
[ -0.04302, 0.12199, -0.021858],
[ -0.041128, -0.0042151, -0.020416],
[ -0.11246, -0.0033688, -0.015662],
[ -0.1236, -0.048412, 0.0064164]])
points_2d: 12x2 numpy array = array([[118, 183],
[147, 159],
[190, 188],
[269, 152],
[220, 130],
[250, 129],
[483, 243],
[445, 247],
[390, 217],
[326, 267],
[384, 303],
[370, 362]])
camera_mat: 3x3 numpy array = array(
[[4018, 0, 2774.2],
[0, 4133.2, 2425],
[0, 0, 1]])
dist_coeffs: 1x5 numpy array = array([[0.23091, -4.0794, 0.0092694, -0.010931, 24.199]])
However, it seems to crash when I get to cv2.solvePnP(), and I have no idea why. I'm using opencv version 4.11.0. I get this exception when it crashes:
Traceback (most recent call last):
File "/Users/gabi/Downloads/BusProjector/main.py", line 60, in <module>
rvec, tvec = Calibrator.solvePnP(img1_3d_points, img_points2_set, calibrator._camera_matrix, calibrator._dist_coeffs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gabi/Downloads/BusProjector/util/calibrator.py", line 149, in solvePnP
success, rvec, tvec = cv2.solvePnP(points_3d, points_2d, camera_mat, dist_coeffs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cv2.error: OpenCV(4.11.0) /Users/xperience/GHA-Actions-OpenCV/_work/opencv-python/opencv-python/opencv/modules/calib3d/src/solvepnp.cpp:824: error: (-215:Assertion failed) ( (npoints >= 4) || (npoints == 3 && flags == SOLVEPNP_ITERATIVE && useExtrinsicGuess) || (npoints >= 3 && flags == SOLVEPNP_SQPNP) ) && npoints == std::max(ipoints.checkVector(2, CV_32F), ipoints.checkVector(2, CV_64F)) in function 'solvePnPGeneric'
The error message is clear enough, elements of points_2d should be float or double.