Background:
For simplicity, I have 3 Cameras. Each one of them is calibrated by its own (its interstice parameters are known and accurate enough)
They all share almost the same view. I want to use this system of 3 Cameras for 3D reconstruction. I have calibrated each pair alone (1 & 2, 2 & 3 and 1 & 3). Thus resulted in 3 transformation matrices. Theoretically, these 3 transformation matrices should has the following propriety:
T13 = T12 * T23
Or in other formula:
T31 * T12 * T23 = Identity
However, practically, this is not true. The result is a bit shifted from the identical matrix due to calibration error.
Error = [T31 * T12 * T23] - [Identity] Eq.3
Problem:
I feel calibrating each pair of cameras alone is such a bad idea. By applying a some kind of global optimization that fuses the whole three pairs together trying to minimizing the error of the equation (3) should give better results.
My question is:
Is there any known previous work on this problem. Any Idea how to use OpenCV to achieve my goal so I do not reinvent the wheel again.
Well, of course sets of many cameras have been calibrated jointly in the past (did you watch the "bullet shot" effect in The Matrix ? This is how it was done by the team at Manex first: https://www.youtube.com/watch?v=uPNBdDNZbYk )
You can use your initial pairwise calibration as the initialization point for a round of bundle adjustment. It is preferable to use a rigid 3D object (not a planar "checkerboard" target, you need a target with actual depth) visible in several cameras in all of the images, and use many images spanning the field of view and the depths of interest. If it's a one-off project, the image matching and bundling can easily be done in a semiautomatic tool with a good GUI. I have satisfactorily used Blender (www.blender.org) for these tasks in the past. Look online for tutorials on using Blender for matchmoving.