I have 4 different views of an object, each with its corresponding K and M camera matrices (intrinsic and extrinsic parameters).
I have 6 3D points which i marked on each of the 2D views (so 24 2d points).
What i am trying to do to back-project those 2D points to 3D and get my 6 3D points in space.
Most papers show how to do it when you have 2 views and their matrices. However, I have 4 views. I am using OpenCV in Python.
As a first approach maybe i could triangulate using all combinations of 2 views, so i get 4choose2 = 6 sets of 3D points and then I average over all triangulations.
Is there a better approach that would be more advisable? Do you think i should use something other than OpenCV? (it still has to be Python though)
I see two solutions for your problem.
1. Analytic Solution
If you do the math, you find that each 2D to 3D correspondence adds 2 equations to the triangulation system. With 6 views you will then get a system with 12 equations and 3 unknowns. This should be the most efficient implementation, but could be unstable when noisy inputs are expected.
2. Triangulate + Optimize
Compute an estimated 3D point with classic 2-View triangulation. Use non-linear optimization to compute the 3D point that minimized the reprojection error to all 6 views. This is my recommend solution, because it should be stable and everything is already implemented in OpenCV.