geometryopen3d

Open3D: How to visualize 2D depth map as 3D?


What I want

I would like to visualize depth map image in Open3D, like the right of below image.

Phillips, Flip & Todd, James & Koenderink, Jan & Kappers, Astrid. (2003). Perceptual representation of visible surfaces. Perception & psychophysics. 65. 747-62. 10.3758/BF03194811. Image source: Phillips, Flip & Todd, James & Koenderink, Jan & Kappers, Astrid. (2003). Perceptual representation of visible surfaces. Perception & psychophysics. 65. 747-62. 10.3758/BF03194811.

What I tried

I converted the depth map image obtained through my preprocessing into a Point Cloud and visualized it. This works fine.

// @image_8uc1 is single channel depth image.
std::shared_ptr<geometry::PointCloud> DepthMap2PointCloud(const cv::Mat& image_8uc1,
  const bool normalized = false) {
  std::shared_ptr<geometry::PointCloud> cloud(new geometry::PointCloud);
  for (int y = 0; y < image_8uc1.rows; y++) {
    const uchar* ptr = image_8uc1.ptr<uchar>(y);
    for (int x = 0; x < image_8uc1.cols; x++) {
      if (normalized) {
        const double _x = ((float)x / (float)image_8uc1.cols);
        const double _y = ((float)y / (float)image_8uc1.rows);
        const double _z = (float)ptr[x] / 255.0;
        Eigen::Vector3d p(_x, _y, _z);
        cloud->points_.push_back(p);
      }
      else {
        Eigen::Vector3d p(x, y, ptr[x]);
        cloud->points_.push_back(p);
      }
    }
  }
  return cloud;
}

// ... 
auto cloud = DepthMap2PointCloud(map);
visualization::Visualizer visualizer;
visualizer.CreateVisualizerWindow("Open3D", 1600, 900);
visualizer.AddGeometry(cloud);
visualizer.Run();
visualizer.DestroyVisualizerWindow();

Result image: enter image description here

Questions

  1. How do I create a surface using the structural properties of my depth map? The distances of (x,y) between all adjacent points are same.
  2. Is there a more effective data structure to solve this problem other than open3d::geometry::PointCloud in Open3D?

Any advices would be save my day.


Solution

  • Using open3d::geometry::TriangleMesh solved the problem.

    std::shared_ptr<geometry::TriangleMesh> DepthMap2Mesh(const cv::Mat& image_8uc1,
      const bool normalized = false) {
      std::shared_ptr<geometry::TriangleMesh> mesh(new geometry::TriangleMesh);
    
      // Push vertices point
      for (int y = 0; y < image_8uc1.rows; y++) {
        const uchar* ptr = image_8uc1.ptr<uchar>(y);
        for (int x = 0; x < image_8uc1.cols; x++) {
    
          // Vertices index
          // w*0+0    w*0+1   w*0+2   w*0+3   w*0+4   ...
          // w*1+0    w*1+1   w*1+2   w*1+3   w*1+4   ...
    
          if (normalized) {
            const double _x = ((float)x / (float)image_8uc1.cols);
            const double _y = ((float)y / (float)image_8uc1.rows);
            const double _z = (float)ptr[x] / 255.0;
            mesh->vertices_.push_back(Eigen::Vector3d(_x, _y, _z));
          }
          else {
            mesh->vertices_.push_back(Eigen::Vector3d(x, y, ptr[x]));
          }
        }
      }
    
      // Push trangle as indices of vertices.
      for (int y = 0; y < image_8uc1.rows - 1; y++) {
        for (int x = 0; x < image_8uc1.cols - 1; x++) {
          const int v0 = image_8uc1.cols * y + x;
          const int v1 = v0 + 1;
          const int v2 = v0 + image_8uc1.cols;
          const int v3 = v2 + 1;
          mesh->triangles_.push_back(Eigen::Vector3i(v0, v1, v2));
          mesh->triangles_.push_back(Eigen::Vector3i(v3, v2, v1));
        }
      }
    
      mesh->ComputeVertexNormals();
      return mesh;
    }
    
    // ... 
    auto mesh= DepthMap2Mesh(map);
    visualization::Visualizer visualizer;
    visualizer.CreateVisualizerWindow("Open3D", 1600, 900);
    visualizer.AddGeometry(mesh);
    visualizer.Run();
    visualizer.DestroyVisualizerWindow();
    

    Result enter image description here

    Applying spline to a square made up of two triangles will be my next goal.