I am successfully streaming a file (audio/video) through UDP on Windows and watching it on another machine with VLC (this was covered on Stackoverflow before):
gst-launch-1.0 -v filesrc location=video.mkv ! decodebin name=dec ! videoconvert ! x264enc ! video/x-h264 ! mpegtsmux name=mux ! queue ! udpsink host=127.0.0.1 port=5000 sync=true dec. ! queue ! audioconvert ! voaacenc ! audio/mpeg ! queue ! mux.
You can test this on VLC: Media > Open Network Stream > Network URL > udp://@:5000
However, while the video is being streamed I would like to also display it on a window, so I could watch the stream myself (no audio needed).
To accomplish this, I started with a series of small experiments so I could change the original pipeline without any surprises. If you are reading this question you know my plan didn't work so well.
My first experiment was to display just the video on single window:
gst-launch-1.0 -v filesrc location=video.mkv ! decodebin ! autovideosink
Then, I changed it to display the same video on 2 windows, to make sure I understood how to work with multithreading:
gst-launch-1.0 -v filesrc location=video.mkv ! decodebin name=dec ! queue ! tee name=t t. ! queue ! videoconvert ! autovideosink t. ! autovideosink
Finally, it came the moment to blend those 2 parts together and stream the video through the network while it is displayed locally. The result is not what I expected, of course: only the first frame appears to be streamed and then everything freezes:
gst-launch-1.0 -v filesrc location=video.mkv ! decodebin name=dec ! tee name=t ! queue ! autovideosink t. ! queue ! videoconvert ! x264enc ! video/x-h264 ! mpegtsmux name=mux ! queue ! udpsink host=127.0.0.1 port=5000 sync=true dec. ! queue ! audioconvert ! voaacenc ! audio/mpeg ! queue ! mux.
It seems that the data is not flowing through the pipeline anymore (for some reason unknown to me) and my attempt to add autovideosink broke everything.
Any tips on how to do this correctly?
The right moment to split the data is right after filesrc:
gst-launch-1.0 -v filesrc location=video.mkv ! tee name=t ! queue ! decodebin ! autovideosink t. ! queue ! decodebin name=dec ! videoconvert ! x264enc ! video/x-h264 ! mpegtsmux name=mux ! queue ! udpsink host=127.0.0.1 port=5000 sync=true dec. ! queue ! audioconvert ! voaacenc ! audio/mpeg ! queue ! mux.
So the data flows directly to autovideosink before anything else happens while the other thread also joins at this very same moment, carrying the data flow to queue and the 2nd decodebin.