I'm currently working on a remotely controlled robot that is sending two camera streams from a Jetson Nano to a PC/Android Phone/VR Headset.
I've been able to create a stable link between the robot and PC using gst-rtsp-server running this pipeline:
./test-launch nvarguscamerasrc sensor-id=1 ! video/x-raw(memory:NVMM) width=1920 height=1080 framerate=30/1 format=NV12 ! nvvidconv flip-method=2 ! omxh264enc iframeinterval=15 ! h264parse ! rtph264pay name=pay0 pt=96
And receiving it on PC using:
gst-launch-1.0 -v rtspsrc location=rtspt://192.168.1.239:8554/test ! application/x-rtp, payload=96 ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false
On PC, there's an excellent latency of about ~120ms, so I thought there wouldn't be a problem running that same thing on Android. Using gstreamer's prebuild binaries from here and a modification from here to be able to use rtspsrc I've succesfully managed to receive the rtsp stream. But this time the video is "slowed down" (probably some buffer problems, or HW acceleration?)
Worked my way around that by using latency=150 drop-on-latency=true parametrs of rtspsrc which only keeps those frames with lower latency but as expected the output encoded image is trash.
So my question is: Why is there such a difference between a phone and a PC receiving the stream.
It seems that the gst-rtsp-stream is defaulting to sending via tcp which i tried to configure with gst_rtsp_media_factory_set_protocols(factory, GST_RTSP_LOWER_TRANS_UDP_MCAST) but doing that I can no longer receive the stream even on a PC with the same pipeline.
Is there a way to force gst-rtsp-server to send via udp. Or is there a way to optimize the phone encoding performance to run as quick a the PC does? (I have an Galaxy S10+, so I guess it should be able to handle that)
My goal is a clear video on Android/VR Headset with minimal latency (preferably the same ~120ms as on PC)
The rtsp server uses TCP because your client query asked for that using rtspt where last t queries for TCP transport. Just using rstp instead should use UDP. You may have a look to protocols property of rtspsrc for more details.
Full story is in the comments here and continued to solution here: Gstreamer Android HW accelerated H.264 encoding