I am trying to stream some video to a browser using WebRTC. To accomplish this, I have a gstreamer instance that encodes the video and sends it to a server via RTP which simply packs it into WebRTC to display. If I use VP8 (vp8enc), this works as expected. However, when using H.264, my browser does not decode any frames:
gst-launch-1.0 videotestsrc ! videoscale ! video/x-raw,width=640,height=360 ! x264enc ! rtph264pay ! udpsink host=127.0.0.1 port=6001 # < my forwarding server
Furthermore, if I run my pipeline on my Khadas VIM4 and use its hardware H.264 encoder (amlvenc) instead of x264 with the exact same pipeline otherwise, the video is displayed (at least in Firefox).
What could be the difference between these two encoders (which both output H.264 video), or how could I find the cause for this problem? I tried looking through the browser WebRTC debugging pages, but this did not really help me...
Most likely the problem is coming from SPS/PPS not being sent with each IDR frame from x264enc, whereas amlvenc most likely does that by default.
There are 2 things you can try adjusting if this is the case:
config-interval=-1 on your rtph264pay. See documentation of config-interval.x264enc to emit SPS/PPS with each IDR frame. It seems that this option is not directly exposed via gstreamer options, but you can try setting the underlying x264 option named repeat-headers via option-string. So try adding option-string=\"repeat-headers=1\" to your x264enc node. See option-string and the relevant x264 option repeat-headersI think either of these should help, if not, then you can also inspect the a=fmtp:96 profile-level-id=42e01f; packetization-mode=1 line in the SDP to see if both encoders use drastically different profiles, but i don't think this should be the case.