serial-portvideo-streaminggstreamerh.264imx6

GStreamer/iMX6: streaming h264 encoded video over serial port between iMX6 and PC

Recently, I have started to work on a project that aims at live video streaming applications based on imx6 processors. A quick description of what I have done so far and what I am trying to do:

  1. Setup: imx6 board(Boundary devices Sabre Lite) acting as the video server(using GStreamer imx plugins), and a PC running Ubuntu on it that receives data from imx6 and streams the video using GStreamer features.
  2. On the imx6 processor, I run the 'testvideosrc' and have successfully streamed it using the RTP over UDP using eternet interface between the imx6 and the PC.
  3. Accessing the serial port using the device files in Linux, next I tried out dumping the video data from imx6 board to a serial port and reading this serial port on the PC. For this, the baud rate of both devices was configured to 115200 baud. The encoding 'bitrate' is configured to 5Kbps. Here are the commands:

IMX6:

#gst-launch-1.0 -v videotestsrc pattern=18 ! video/x- raw,width=100,height=50 ! imxvpuenc_h264 bitrate=5 ! h264parse ! filesink location=/dev/ttyUSB1

PC:

#CAPS=video/x-h264 
#gst-launch-1.0 -v filesrc location=/dev/ttyUSB1 ! $CAPS ! h264parse ! avdec_h264 ! autovideosink sync=true

There are no errors observed on the imx6 board. However, I see the following errors at the PC side: 

# GST_DEBUG=3 gst-launch-1.0 -v filesrc location=/dev/ttyUSB1 ! $CAPS ! h264parse ! avdec_h264 ! autovideosink sync=true 
  Setting pipeline to PAUSED … 
  0:00:00.066439392 15475 0x556d8a01d160 WARN basesrc gstbasesrc.c:3583:gst_base_src_start_complete: pad not activated yet 
  Pipeline is PREROLLING … 
  0:00:21.730466251 15475 0x556d8a000940 WARN capsfilter 
  gstcapsfilter.c:455:gst_capsfilter_prepare_buf: error: Filter caps do not completely specify the output format 
  0:00:21.730523691 15475 0x556d8a000940 WARN capsfilter gstcapsfilter.c:455:gst_capsfilter_prepare_buf: error: Output caps are unfixed: video/x-h264, width=(int)[ 1, 8192 ], height=(int)[ 1, 8192 ], framerate=(fraction)[ 0/1, 2147483647/1 ] 
  0:00:21.730676173 15475 0x556d8a000940 WARN basetransform gstbasetransform.c:2159:default_generate_output: could not get buffer from pool: error 
  0:00:21.730742223 15475 0x556d8a000940 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop: error: Internal data stream error. 
  0:00:21.730775478 15475 0x556d8a000940 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop: error: streaming stopped, reason error (-5) 
  ERROR: from element /GstPipeline:pipeline0/GstCapsFilter:capsfilter0: Filter caps do not completely specify the output format 
  Additional debug info: 
  gstcapsfilter.c(455): gst_capsfilter_prepare_buf (): /GstPipeline:pipeline0/GstCapsFilter:capsfilter0: 
  Output caps are unfixed: video/x-h264, width=(int)[ 1, 8192 ], height=(int)[ 1, 8192 ], framerate=(fraction)[ 0/1, 2147483647/1 ] 
  ERROR: pipeline doesn’t want to preroll. 
  Setting pipeline to NULL … 
  Freeing pipeline …

  1. Since, the encoding rate is 5Kbps (bitrate=5, as specified in the above command), I suppose it is possible to send this amount of data via the serial port. I realize that currently, the caps negotiation fails, however, I am unsure of how to proceed with this.

  2. On the PC side, reading the serial port with 'cat /dev/ttyUSB1' succeeds with limited data. The data is unreadable(as expected), however it is not a continuous stream.

Does anyone have an idea, on how to solve this. I also think, I am misinterpreting the usage of Linux device files, when I am attempting to read over the serial data file using GStreamer.

My later test, would be to use an actual camera(MIPI) and try streaming it over serial port. Does it seem feasible or is a completely crazy idea to do?

Solution

  • With the following commands, I could get this working on serial with a baud of 19200. However, the latency is very high in the range of 5-6 seconds. With a baud of 1M, it works with less noticeable latency < 1s.

    imx6:

    gst-launch-1.0 -v videotestsrc pattern=18 ! video/x-raw,width=100,height=50
! imxvpuenc_h264 bitrate=5 ! h264parse ! filesink location=/dev/ttyUSB0
blocksize=1024 max-bitrate=19000 sync=false

    PC:

    gst-launch-1.0 -v filesrc location=/dev/ttyUSB1 blocksize=1024 ! $CAPS !
h264parse ! avdec_h264 lowres=2 skip-frame=0 ! autovideosink sync=false