[SOLVED] How to transform a 3D bounding box from pointclouds to matched RGB frames in ROS?

How to transform a 3D bounding box from pointclouds to matched RGB frames in ROS?

i am doing SLAM with depth camera realsense D455 on ROS (https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i) and I am able to create 3D point clouds of the environment. I am new in ROS and my goal is to bound a 3d box in a region of global pointcloud then transform the same box on RGB frames that are matched with that points(at the same coordinates) in the global point cloud. now i have RGB, depth, and point cloud topic and tf but since i am new in this field i do not know how can i find RGB frames that are matched with each point in the global point cloud? and how do the same operation from point cloud to RGB frames? i would be grateful if someone can help me with that

Solution

Get 3D bounding box points.
Transform them into camera's frame (z is depth).
Generate your camera's calibration model (intrinsic and extrinsic parameters of the camera, focal length etc.) -> Check openCV model, Scaramuzza Model or another stereo camera calibration model
Project 3D points to 2D pixels. (openCV has one projectPoints() function).