We are relatively new to GStreamer. We are running GStreamer 1.18.4 on a Linux-based custom board and we have a pipeline receiving video from an IP camera using rtspsrc and creating MP4 video clips of 10 seconds. We faced the missing PTS issue and we were able to solved it by implementing the workaround of turning-on the interpolation from our C code (gst_base_parse_set_pts_interpolation on). Now the 10 secs video clips are properly recorded.
The pipeline we are using to create the videos is the following (with the interpolation set to on for the element h264parse):
gst-launch-1.0 rtspsrc user-id="admin" user-pw="admin"
location="rtsp://192.168.0.131:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif" !
rtph264depay ! h264parse !
splitmuxsink location=video_%02d.mp4 max-files=400 max-size-time=100000000
Now we are facing an issue in which we need to concatenate some of those videos(let's say concatenate 6 10secs videos to get a long 1-minute video) and our pipeline often gets broken.
We believe the pipeline gets broken when one of the videos being concatenated has PTS interpolated (the workaround we implemented).
The pipeline we are using to concat is:
gst-launch-1.0 concat name=c ! queue ! m.video_0
qtmux name=m ! filesink location=/data/users/videos/output_video.mp4
filesrc location=/data/users/videos/cam1_input_video_1.mp4 ! qtdemux ! h264parse ! c.
filesrc location=/data/users/videos/cam2_input_video_2.mp4 ! qtdemux ! h264parse ! c.
We have tested the pipeline concatenating multiple times the same video and in that case, everything works as expected. Fails (sometimes) when concatenating different files.
We have tried several different modifications to the pipeline, but all of them fail on the qtdemux element with the following message:
gst-launch-1.0 --gst-debug-level=3 concat name=c ! queue ! m.video_0 qtmux name=m ! filesink location=/data/users/videos/output_video.mp4 filesrc location=/data/users/videos/cam1_input_video_1.mp4 ! qtdemux ! h264parse ! c. filesrc location=/data/users/videos/cam1_input_video_2.mp4 ! qtdemux ! h264parse ! c.
Setting pipeline to PAUSED ...
0:00:00.461273255 10102 0x60c030 WARN aggregator gstaggregator.c:2046:gst_aggregator_query_latency_unlocked:<m> Latency query failed
0:00:00.465287035 10102 0x6210a0 WARN basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<filesrc2> pad not activated yet
0:00:00.466665891 10102 0x6210a0 WARN basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<filesrc1> pad not activated yet
0:00:00.467579600 10102 0x6210a0 WARN basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<filesrc0> pad not activated yet
Pipeline is PREROLLING ...
0:00:00.471999988 10102 0x60be60 WARN qtdemux qtdemux_types.c:245:qtdemux_type_get: unknown QuickTime node type pasp
0:00:00.472496716 10102 0x60be60 WARN qtdemux qtdemux.c:3067:qtdemux_parse_trex:<qtdemux2> failed to find fragment defaults for stream 1
0:00:00.475729572 10102 0x60bdb0 WARN qtdemux qtdemux_types.c:245:qtdemux_type_get: unknown QuickTime node type pasp
0:00:00.476105597 10102 0x60bdb0 WARN qtdemux qtdemux.c:3067:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 1
0:00:00.480323577 10102 0x60bef0 WARN qtdemux qtdemux_types.c:245:qtdemux_type_get: unknown QuickTime node type pasp
0:00:00.480799931 10102 0x60bef0 WARN qtdemux qtdemux.c:3067:qtdemux_parse_trex:<qtdemux1> failed to find fragment defaults for stream 1
0:00:00.525340747 10102 0x60c030 FIXME basesink gstbasesink.c:3386:gst_base_sink_default_event:<filesink0> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:00.526037550 10102 0x60c030 WARN qtmux gstqtmux.c:3077:gst_qt_mux_start_file:<m> Robust muxing requires reserved-moov-update-period to be set
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:00.529879629 10102 0x60c030 FIXME aggregator gstaggregator.c:1367:gst_aggregator_aggregate_func:<m> Subclass should call gst_aggregator_selected_samples() from its aggregate implementation.
New clock: GstSystemClock
0:00:00.770191724 10102 0x60c090 WARN qtmux gstqtmux.c:5346:gst_qt_mux_can_renegotiate:<m> pad video_0 refused renegotiation to video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3.1, profile=(string)main, codec_data=(buffer)014d401fffe1002e674d401f8d8d402802dff80b7010101400000fa000027103a1801e9c0003567ebbcb8d0c00f4e0001ab3f5de5c2801000468ee3880, width=(int)1280, height=(int)720, framerate=(fraction)20000/1001, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
0:00:00.782437264 10102 0x60c090 WARN qtmux gstqtmux.c:5346:gst_qt_mux_can_renegotiate:<m> pad video_0 refused renegotiation to video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3.1, profile=(string)main, codec_data=(buffer)014d401fffe1002e674d401f8d8d402802dff80b7010101400000fa000027103a1801e9c0003567ebbcb8d0c00f4e0001ab3f5de5c2801000468ee3880, width=(int)1280, height=(int)720, framerate=(fraction)20000/1001, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
0:00:00.783267561 10102 0x60c090 WARN qtmux gstqtmux.c:5346:gst_qt_mux_can_renegotiate:<m> pad video_0 refused renegotiation to video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3.1, profile=(string)main, codec_data=(buffer)014d401fffe1002e674d401f8d8d402802dff80b7010101400000fa000027103a1801e9c0003567ebbcb8d0c00f4e0001ab3f5de5c2801000468ee3880, width=(int)1280, height=(int)720, framerate=(fraction)20000/1001, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
0:00:00.784103274 10102 0x60c090 WARN qtmux gstqtmux.c:5346:gst_qt_mux_can_renegotiate:<m> pad video_0 refused renegotiation to video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3.1, profile=(string)main, codec_data=(buffer)014d401fffe1002e674d401f8d8d402802dff80b7010101400000fa000027103a1801e9c0003567ebbcb8d0c00f4e0001ab3f5de5c2801000468ee3880, width=(int)1280, height=(int)720, framerate=(fraction)20000/1001, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
0:00:00.785129478 10102 0x60c090 WARN qtmux gstqtmux.c:5346:gst_qt_mux_can_renegotiate:<m> pad video_0 refused renegotiation to video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3.1, profile=(string)main, codec_data=(buffer)014d401fffe1002e674d401f8d8d402802dff80b7010101400000fa000027103a1801e9c0003567ebbcb8d0c00f4e0001ab3f5de5c2801000468ee3880, width=(int)1280, height=(int)720, framerate=(fraction)20000/1001, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
0:00:00.793758637 10102 0x60c090 WARN qtmux gstqtmux.c:5346:gst_qt_mux_can_renegotiate:<m> pad video_0 refused renegotiation to video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3.1, profile=(string)main, codec_data=(buffer)014d401fffe1002e674d401f8d8d402802dff80b7010101400000fa000027103a1801e9c0003567ebbcb8d0c00f4e0001ab3f5de5c2801000468ee3880, width=(int)1280, height=(int)720, framerate=(fraction)20000/1001, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
0:00:00.794520312 10102 0x60bef0 WARN qtdemux qtdemux.c:6545:gst_qtdemux_loop:<qtdemux1> error: Internal data stream error.
0:00:00.794615432 10102 0x60bef0 WARN qtdemux qtdemux.c:6545:gst_qtdemux_loop:<qtdemux1> error: streaming stopped, reason not-negotiated (-4)
ERROR: from element /GstPipeline:pipeline0/GstQTDemux:qtdemux1: Internal data stream error.
Additional debug info:
../gst-plugins-good-1.18.4/gst/isomp4/qtdemux.c(6545): gst_qtdemux_loop (): /GstPipeline:pipeline0/GstQTDemux:qtdemux1:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.605594272
Setting pipeline to NULL ...
Freeing pipeline ...
Any idea how we can improve the pipeline to make the concatenation better and more robust against possible errors/inconsistencies of the PTS?
We were able to solve the issue by replacing the concat element with splitmuxsrc.
The final pipeline we used is:
splitmuxsrc name=splitsrc ! h264parse ! qtmux ! filesink name=sink
The list of files to be concatenated is passed from C to the splitsrc element through the format-location signal:
GStrv *format_location_callback (GstElement * splitmux,
gpointer udata)