pythonopencvcomputer-visionstereo-3d

Steps for 3D Reconstruction?


This question has been asked in different ways many times, many people approach this problem in different ways. Which is why I am unsure of my methodology, it would be good if someone could tell me if I am doing things correctly.

The steps that I follow are -

  1. Set up stereo camera pair.
  2. Calibrate both the cameras individually to get their camera matrices, distortion coefficients, using calibrateCamera
  3. Then we calculate R, T, E, F using stereoCalibrate
  4. Calculate R1,R2,P1,P2 and Q using stereoRectify
  5. Since I want to find the 3D coordinates of a specific point in my scene, I extract the coordinates of that point in both the images, and after that I use triangulatePoints to get the 3D points in homogenous coordinates.

Are the steps that I'm following correct? I've been having problems with the OpenCV documentation so it took me a while to formulate the steps, digging through the documentation. People have approached this problem in OpenCV in different ways, some have even constructed the Fundamental and Essential Matrices using some of the helper functions in the OpenCV documentation, but I think the stereoCalibrate and stereoRectify use these helper functions themselves.


Solution

  • As you said, everyone has their own way. The way I tried is, the first 4 steps were the same, but for calculating the 3d point I used the stereo matching algorithms that are present in OpenCV. Algorithms like StereoBM, StereoSGBM are implemented in OpenCV and it has provided an easy to use APIs.

    So Once we get the disparity map from the stereo matching funciton, we can get the 3d coordinates by using reprojectImageTo3D API. It requires disparity map and Q matrix as inputs.