c++video-streamingstreaminggstreamermultimedia

How can I get frame by using Gstreamer?


I`m a beginner at using Gstreamer to handle some input videos. I have already built the pipeline using GStreamer to transcode the videos but the last part I cannot do is How I can get those batches of frames and do some custom image processing techniques to handle the purpose of my task.


Input Videos -----> Gstreamer Pipeline -----> Task: Apply some Image Processing Techniques

I`ve been searching about this problem on the Internet but cannot find any solution and the more I search, the more I am confused.


Solution

  • AppSink is the good element for you. You can enable "emit-signal" property and listen the event "new-sample". Then you can get an access to the buffer.

    Here the entire documentation :

    https://gstreamer.freedesktop.org/documentation/tutorials/basic/short-cutting-the-pipeline.html?gi-language=c

    You have to create appsink element, enable "emit-signals" then register "new-sample" callback like this :

    g_signal_connect (data.app_sink, "new-sample", G_CALLBACK (new_sample), &data)
    
    
    static GstFlowReturn new_sample (GstElement *sink, CustomData *data) {
      GstSample *sample;
      /* Retrieve the buffer */
      g_signal_emit_by_name (sink, "pull-sample", &sample);
      if (sample) {
        /* The only thing we do in this example is print a * to indicate a received buffer */
        g_print ("*");
        gst_sample_unref (sample);
        return GST_FLOW_OK;
      }
      return GST_FLOW_ERROR;
    }
    

    Now you can retrieve buffer from sample instead of g_print ... (gst_sample_get_buffer)

    https://gstreamer.freedesktop.org/documentation/gstreamer/gstsample.html?gi-language=c

    Then read data inside the buffer :

      GstMapInfo info;
      gst_buffer_map (buf, &info, GST_MAP_READ);
      
      gst_buffer_unmap (buf, &info);
      gst_buffer_unref (buf);
    

    info.data ==> buffer content.

    Best regards.