Rendering depth from mesh and camera parameters

I want to render depth from mesh file, and camera parameters, to do so I tried RaycastingScene from open3d with this mesh file as follows:

#!/usr/bin/env python3

import numpy as np
import open3d as o3d
import matplotlib.pyplot as plt


def render_depth(
    intrins:o3d.core.Tensor,
    width:int,
    height:int,
    extrins:o3d.core.Tensor,
    tmesh:o3d.t.geometry.TriangleMesh
)->np.ndarray:
    """
    Render depth from mesh file

    Parameters
    ----------
    intrins : o3d.core.Tensor
        Camera Intrinsics matrix K: 3x3
    width : int
        image width
    height : int
        image height
    extrins : o3d.core.Tensor
        camera extrinsics matrix 4x4
    tmesh : o3d.t.geometry.TriangleMesh
        TriangleMesh

    Returns
    -------
    np.ndarray
        Rendred depth image
    """
    scene = o3d.t.geometry.RaycastingScene()
    scene.add_triangles(tmesh)
    
    rays = scene.create_rays_pinhole(
        intrinsic_matrix=intrins,
        extrinsic_matrix=extrins,
        width_px=width, height_px=height
    )
    
    ans = scene.cast_rays(rays)
    t_hit = ans["t_hit"].numpy() / 1000.0

    return t_hit


if __name__=="__main__":
    import os
    mesh_path = f"{os.getenv('HOME')}/bbq_sauce.ply"
    mesh = o3d.t.io.read_triangle_mesh(mesh_path)
    mesh.compute_vertex_normals()
    
    # camera_info[k].reshape(3, 3)
    intrins_ = np.array([
            [606.9275512695312, 0.0, 321.9704895019531],
            [0.0, 606.3505859375, 243.5377197265625],
            [0.0, 0.0, 1.0]
            ])
    width_  = 640 # camera_info.width
    height_ = 480 # camera_info.height
    # root2cam 4x4
    extrens_ = np.eye(4)
    #
    intrins_t = o3d.core.Tensor(intrins_)
    extrins_t = o3d.core.Tensor(extrens_)
    
    rendered_depth = render_depth(
        intrins=intrins_t, 
        width=width_, 
        height=height_, 
        extrins = extrins_t, 
        tmesh=mesh
    )
    
    plt.imshow(rendered_depth)
    plt.show()

but I'm getting a depth image which doesn't seem to be correct!

Can you please tell me how can I fix that? thanks.

The raycasting scene is working correctly. However, your extrinsic matrix is setting the camera inside the mesh and thus you are seeing how mesh looks from inside.

I roughly measured the mesh bbq_sauce in meshlab and it is of width ~53 units. You can either move the camera away from the mesh, using either:

extrens_[2, 3] = 200

or instead of moving the object, just move the camera pose.

rays = o3d.t.geometry.RaycastingScene.create_rays_pinhole(
        fov_deg=90,  # Not computed from intrinsic parameters.
        center=[0, 0, 0],  # Look towards
        eye=[100, 100, 0],  # Look from
        up=[0, 1, 0],  # This helps in orienting the camera so object is not inverted.
        width_px=640,
        height_px=480,
    )