pythonc++opencvraspberry-pidisparity-mapping

What is the best way(algorithim or function) of generating a decent disparity map in c++ or python for implementation on raspberry pi?


I am trying to do some distance calculations using stereo videos in realtime but there is no proper documentation available for disparity mapping for opencv3 and many of the old commands from opencv2 are not working on the latest versions. Is there any other way (library or algorithm) to do disparity mapping in c++ or python.

I am trying to implement my code on a raspberry pi and performance is an issue for me. I know that c++ is much faster and relatively less processor intensive so I might use that but any help would be appreciated in this regard.


Solution

  • Yes, but if you want anything reasonably fast or accurate, you'll want to use semi-global matching powered by CUDA: https://github.com/fixstars/libSGM . Using a library like this, you can get realtime disparity maps for images of ~640x480 pixels.

    The first step will be to calibrate your imaging system/stereo camera using OpenCV. There are many, many tutorials on this, but this is a good place to start: https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html

    Then once calibrated and once your rectification matrices are created, you will need to rectify your incoming images so that the they are in a rectified coordinates, and any distortion due to the lenses or other defects in the optical system will be corrected for.

    Then use your desired disparity algorithm/library to generate a disparity map from this rectified pair. This will generally be between 128 and 256 disparity levels, the latter being less common. You can then convert this disparity map to a metric depth map by using cv::reprojectImageTo3D(). From here you can fuse the depth maps using any number of fusion or SLAM libraries, KinectFusion being one of the oldest and easiest to get into. In fact, it's been integrated into PCL.