unity-game-engineagora.io

How to use scene camera with Agora.io in Unity


In Unity I have integrated Agora.io such that from within my virtual reality app, i can connect a video call to an outside user on a webpage. The VR user can see the website user, but the website user cannot see the VR user because there is no available physical camera to use. Is there a way to use a scene camera for the Agora video feed? This would mean that the website user would be able to see into the VR user's world


Solution

  • Yes. Although I haven't done projects in VR before, but the concept should be there. You may use the External Video Source to send any frames of the video as if it is sent from the physical camera. For Scene cameras, you may use a RenderTexture to output the camera feed, and extract the raw data from the RenderTexture. So the steps are:

    1. Set up your camera to output to a RenderTexture (plus logic to display this RenderTexture somewhere locally if needed.)
    2. Also make sure when you set up the Agora RTC engine, enable external video source using this call:

      mRtcEngine.SetExternalVideoSource(true, false);

    3. At each frame, extract the raw image data from the RenderTexture

    4. Send the raw frame data to the SDK function rtc.pushVideoFrame()

    You may find the code for the last step here https://gist.github.com/icywind/92053d0983e713515c64d5c532ebee21