vlcv4l2pulseaudio

Low fps (choppy video), no sound, "No JPEG data found in image" when using webcam/hdmi-usb dongle


This took me too long to figure out, so in the hopes of helping anyone out there dealing with any of these issues, I wanted to post the solution. But first, the problems:

I bought one of those cheap HDMI->USB dongles and connected my PS3 as a video source. On vlc the image looked crisp, but I was getting no sound, and the video was really choppy. Checking the codec tab in the "info" section, I saw I was getting 1080p at 5 fps. I thought I got a defective dongle, but decided to check with other apps. tvtime/xawtv gave me great framerate, but low resolution that I couldn't change, cheese allowed me to set all the options, and I was getting good framerate, and good resolution (but no sound), and then I finally tried obs which gave me a perfect result. So clearly the dongle is fine, and the problem was with vlc.

See my answer below for the solution to all those problems (and more!)


Solution

  • I found, through much research and experimentation, that the reason the video was choppy in vlc was because it was using the default "chroma" of YUV2, which if I am not mistaken is uncompressed. (You can check your webcam/dongle's capabilities by running: v4l2-ctl --list-formats-ext -d /dev/video0 where /dev/video0 is your device)

    The correct setting to overcome this is mjpg. However, that results in a flood of errors saying:

    [mjpeg @ 0x7f4e0002fcc0] No JPEG data found in image
    

    This is caused by the fact that the default resolution and framerate (1080p@60fps) overwhelm what I guess is the mjpeg decoder. Setting it to 720p, or lowering the framerate to 30fps prevents the errors.

    Next, the sound was missing, and this is due to the fact that I am using pulseaudio and vlc cannot figure out which source to use.

    I found the pulse source by running:

    pactl list short sources
    

    which yielded:

    alsa_input.usb-MACROSILICON_USB_Video-02.multichannel-input
    

    You can test that this is the correct source by running:

    vlc   pulse://alsa_input.usb-MACROSILICON_USB_Video-02.multichannel-input
    

    I found that to combine the v4l2 video source with the correct pulseaudio sink, you have to set the audio via the input-slave parameter to vlc, but unfortunately, that did not work for me as specified in the guides, and instead I had to set the video source as the slave. The final commands that worked for me were either of:

    720p:

    vlc   pulse://alsa_input.YOUR-SOURCE-HERE-input  --input-slave=v4l2:///dev/video0:chroma=mjpg:width=1280:height=720
    

    1080p@30fps:

    vlc   pulse://alsa_input.YOUR-SOURCE-HERE-input  --input-slave=v4l2:///dev/video0:chroma=mjpg:fps=30