augmented-realityslam-algorithm

Visual SLAM for augmented Reality


I'm currently trying to build an Desktop AR application, using the eyewear Vuzix star 1200 XLD and c++. I've made a lot of research regarding the algorithms used in AR and by famous SDK, and I found that visual SLAM is a common used algorithm.

So, I'd like to have some feedback if you used it, and i'm open to any suggestions regarding other AR algorithms used for tracking and matching.

Thanks a lot!


Solution

  • On year later...

    SLAM is not an algorithm, it's an approach, a decomposition in steps to reach the goal. Each step let you choose one of many suitable algorithms, each of them with diverse characteristics, like performance, accuracy, an so on.

    SLAM can be Visual monocular SLAM (monoSLAM), a specializated branch of SLAM, related to visual odometry.

    You can start with feature detector and descriptor extractor algorithm, like FAST & BRIEF. More of the algorithm do both things, like ORB and AKAZE.

    So, you extract and descibe a point cloud from an image, from a frame.

    Then you match points from two (let say consecutive) frames, with a matcher, like a brute force Hamming distance matcher, obtaining something like velocity vectors.

    https://www.youtube.com/watch?v=G8XAHLJR8A4

    Then you apply some algorithm like PnP solver, to get the rototranslation matrix that explain the relative camera motion. Now you have a 3D position vector for each feature.

    You choose landmarks from features detected, start building a 3d points cloud base map, each with its error distribution. Then you apply some bundle adjustment like sba.

    This is the low level starting point. You'll want to build a 3D world with texture, you'll want to close paths accurately, you'll want robustness against moving parts (like peolple, vehicles)...

    All you have dreamed of, is here waiting to be develop by yourself :)