opencvgstreamermat

Convert GstVideoFrame to Opencv Mat and vice versa


I'm extracting a frame into Opencv Mat from GstVideoFrame. Then processing the frame in Mat.

custom_transform_frame_ip (GstVideoFilter *filter, GstVideoFrame *frame)
{
/*** convert GstVideoFrame to opencv format ***/
  GstMapInfo map_info;
  Mat cvframe;

  if (!gst_buffer_map ((frame->buffer), &map_info, GST_MAP_READ)) {
      gst_buffer_unmap ((frame->buffer), &map_info);
      cout << "gst bufer map failed" << endl;
      return GST_FLOW_ERROR;
  }

  cout << "frame width: "<< frame->info.width << "height: " << frame->info.height << endl;

  //cvframe = Mat::zeros(/*1080*/frame->info.width, /*1920*/frame->info.height, CV_8UC3);
  
  cvframe = cv::Mat(/*1080*/frame->info.width, /*1920*/frame->info.height, CV_8UC3, (char *)map_info.data, cv::Mat::AUTO_STEP);
  

  /*** drawing custom polygon ROI on the Mat frame ***/

  /***************************************************/

  /****** convert Mat frame back to GstVideoFrame ******/
  /*memcpy (map_info.data, cvframe.data, map_info.size);

  gst_buffer_map(frame->buffer, &map_info, GST_MAP_WRITE);*/

  /**************************************************/

  gst_buffer_unmap ((frame->buffer), &map_info);

  return;
}

Now I want to put this Mat frame back into GstVideoFrame so that further flow of the pipeline won't be affected.

I've tried to memcpy Mat frame.data into GstVideoFrame->GstVideoInfo.Data field. It's not working out. I'm sure this is wrong approach.

I want to know 2 things:

  1. If I'm correct on the first part i.e. extracting frame from GstVideoFrame to Opencv Mat.

  2. How to put this Mat frame into the same GstVideoFrame structure from where I extracted it. Or I can create another GstVideoFrame and copy the same data with processed Mat frame and then pass further.


Solution

  • Turns out that we're already accessing the pixel data in-place. So no need to copy data somewhere after processing. GstVideoFrame will have the processed data after operating it through Mat.