UPDATE 2018.03.12: The code has been updated and now runs as expected
I am struggeling a bit with creating a GTK+ Window equivalent to the following gst-launch command
gst-launch-1.0 udpsrc port=6000 caps="application/x-rtp" ! rtph264depay ! h264parse ! omxh264dec ! autovideosink
My code is as following:
int main (int argc, char **argv)
{
GdkWindow *video_window_xwindow;
GtkWidget *window, *video_window;
gulong embed_xid;
GstStateChangeReturn sret;
gst_init (&argc, &argv);
gtk_init (&argc, &argv);
GstElement *pipeline, *udpsrc, *appxrtp, *depay, *parse, *omxh264dec, *videoConvert, *sink;
pipeline = gst_pipeline_new ("xvoverlay");
udpsrc = gst_element_factory_make ("udpsrc", NULL); g_assert(udpsrc);
//Set CAPS
g_object_set (G_OBJECT (udpsrc), "port", 6000, NULL);
GstCaps * xrtpcaps = gst_caps_from_string("application/x-rtp,encoding-name=H264");
g_object_set (udpsrc, "caps", xrtpcaps, NULL);
depay = gst_element_factory_make ("rtph264depay", NULL); g_assert(depay);
parse = gst_element_factory_make ("h264parse", NULL); g_assert(parse);
omxh264dec = gst_element_factory_make ("omxh264dec", NULL); g_assert(omxh264dec);
videoConvert = gst_element_factory_make ("videoconvert", NULL); g_assert(videoConvert);
sink = gst_element_factory_make ("xvimagesink", NULL); g_assert(sink);
//ADD
gst_bin_add_many (GST_BIN (pipeline), udpsrc, depay, parse, omxh264dec, videoConvert, sink, NULL);
//LINK
g_assert(gst_element_link (udpsrc, depay));
g_assert(gst_element_link (depay, parse));
g_assert(gst_element_link (parse, omxh264dec));
g_assert(gst_element_link (omxh264dec, videoConvert));
g_assert(gst_element_link (videoConvert, sink));
/* prepare the ui */
window = gtk_window_new (GTK_WINDOW_TOPLEVEL);
g_signal_connect (G_OBJECT (window), "delete-event", G_CALLBACK (window_closed), (gpointer) pipeline);
gtk_window_set_default_size (GTK_WINDOW (window), 600, 400);
gtk_window_set_title (GTK_WINDOW (window), "GstVideoOverlay Gtk+ demo");
video_window = gtk_drawing_area_new ();
gtk_container_add (GTK_CONTAINER (window), video_window);
gtk_container_set_border_width (GTK_CONTAINER (window), 2);
gtk_widget_show_all (window);
video_window_xwindow = gtk_widget_get_window (video_window);
embed_xid = GDK_WINDOW_XID (video_window_xwindow);
gst_video_overlay_set_window_handle (GST_VIDEO_OVERLAY (sink), embed_xid);
/* run the pipeline */
sret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
if (sret == GST_STATE_CHANGE_FAILURE)
gst_element_set_state (pipeline, GST_STATE_NULL);
else
gtk_main ();
gst_object_unref (pipeline);
return 0;
}
When running it I get a black/empty GTK Window.
Console output is:
NvMMLiteOpen : Block : BlockType = 261
TVMR: NvMMLiteTVMRDecBlockOpen: 7907: NvMMLiteBlockOpen
NvMMLiteBlockCreate : Block : BlockType = 261
TVMR: cbBeginSequence: 1223: BeginSequence 320x240, bVPR = 0
TVMR: LowCorner Frequency = 100000
TVMR: cbBeginSequence: 1622: DecodeBuffers = 17, pnvsi->eCodec = 4, codec = 0
TVMR: cbBeginSequence: 1693: Display Resolution : (320x240)
TVMR: cbBeginSequence: 1694: Display Aspect Ratio : (320x240)
TVMR: cbBeginSequence: 1762: ColorFormat : 5
TVMR: cbBeginSequence:1776 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1904: SurfaceLayout = 3
TVMR: cbBeginSequence: 2005: NumOfSurfaces = 24, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 2007: BeginSequence ColorPrimaries = 2, TransferCharacteristics = 2, MatrixCoefficients = 2
Allocating new output: 320x240 (x 24), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 320, nFrameHeight = 240
I am running this as the video source (which seems fine, I am able to consume this with gst-launch without any issues)
gst-launch-1.0 videotestsrc ! omxh264enc control-rate=2 bitrate=8000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! rtph264pay mtu=1400 ! udpsink host=127.0.0.1 port=6000
Can anyone give me some pointers? Do I need to link differently (when pads are added)?
Adding videoconvert between omxh264dec and sink fixed the problem. The video is now rendered in the GTK Window. Have updated the original code.