I'm experimenting the capabilities of VLC-Unit for various video streaming protocol, the simplest of which should be playing the video feed of a USB camera directly.
For this purpose, I start with a simple example implementation (compiled and executed under Windows):
This demo was written to play a video on GoogleDrive using http protocol:
if(_mediaPlayer.Media == null)
{
// playing remote media
_mediaPlayer.Media = new Media(new Uri("http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"));
}
I made the following change to repurpose it for the USB camera:
if (_mediaPlayer.Media == null)
{
// playing remote media
// _mediaPlayer.Media = new Media(new Uri("http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"));
_mediaPlayer.Media =
new Media(
new Uri("dshow://")
);
_mediaPlayer.Media.AddOption(":dshow-vdev=<My USB Camera Name>);
Task.Run(async () =>
{
var success = await _mediaPlayer.PlayAsync();
uint height = 0;
uint width = 0;
_mediaPlayer.Size(0, ref width, ref height);
Debug.Log("Media size: " + width + "x" + height);
}
);
}
Unfortunately, when the example is executed, I got the following debug info:
Media size: 0x0
Obviously, no video could be displayed in this case (audio feed seems to be normal). This phenomenon can be observed even if <My USB Camera Name>
was single or double quoted. What may have caused the VLC-Unity to fail on such simple case? How to fix it?
UPDATE 1: The camera is fully functional when being opened from within VLC player, the log snippet (debug-level verbosity) is show here:
main debug: Creating an input for 'dshow://'
main debug: requesting art for new input thread
main debug: using timeshift granularity of 50 MiB
main debug: using timeshift path: C:\Users\pengc\AppData\Local\Temp
main debug: `dshow://' gives access `dshow' demux `any' path `'
main debug: creating demux: access='dshow' demux='any' location='' file='(null)'
main debug: looking for access_demux module matching "dshow": 15 candidates
main debug: looking for meta fetcher module matching "any": 1 candidates
dshow debug: dshow-vdev: <Camera Name>
dshow debug: dshow-adev: none
Update 1: I guess is not a very specific example. So I uploaded 2 minimalistic experiments on Github. Both of the following experiments were conducted on Windows 11, with both VLC player & OBS Studio installed (so OBS Virtual Camera is always the default capture device)
https://github.com/tribbloid/VLCUnitySpike/blob/main/Assets/Scripts/VLCMinimalPlaybackOBS.cs
the media player was created by:
_mediaPlayer.Media = new Media(
new Uri(
"dshow://"
)
);
_mediaPlayer.Media.AddOption(":dshow-vdev=\"OBS Virtual Camera\"");
_mediaPlayer.Media.AddOption(":dshow-adev=None");
It turns out that the Unity application can only play the first frame, displayed video immediately freeze despite continuous buffer updating. Regardless, it has found the correct camera.
https://github.com/tribbloid/VLCUnitySpike/blob/main/Assets/Scripts/VLCMinimalPlaybackWebcam.cs
Similar to the first example, except the following lines:
_mediaPlayer.Media = new Media(
new Uri(
"dshow://"
)
);
_mediaPlayer.Media.AddOption(":dshow-vdev=\"USB2.0 HD UVC WebCam\"");
_mediaPlayer.Media.AddOption(":dshow-adev=None");
The outcome is identical to Experiment 1, the same default OBS camera was used, and only the first frame was displayed. I haven't tested when all arguments are removed but the result should be identical.
This repository can be opened by any Unity Editor 2022 LTS, the VLCUnity asset however has to be manually imported. I imported the proprietary release but the trial release should work equally well.
So far the cause for this problem is still unknown. I am keen to know if others can replicate it.
UPDATE 2: I can upload the trial release of the asset (which should have an open-source license) for easily reproducing my problem.
Found the problem: there is no need to quote the video capture device name in the API, because each string can only contain 1 argument, and there is no need to escape space.
The correct program snippet is:
_mediaPlayer.Media.AddOption(":dshow-vdev=USB2.0 HD UVC WebCam");
_mediaPlayer.Media.AddOption(":dshow-adev=None");
The OBS camera issue is a known bug, unfortunately it cannot be used until a later version