unity-game-engineaugmented-realityvuforiagesture-recognitionandroid-augmented-reality

Using MediaPipe Hand Tracking with Unity


We are working on a project for a Mobile (Android) Mixed-Reality (MR) game using only hand gestures for interacting with the virtual objects. We've researched for frameworks and api's that we could use for the project. Currently, we think the best way is to use Google Media Pipe for Multihand tracking and Vuforia for creating the MR environment. But since Vuforia is used with Unity and MediaPipe does not have a Unity port yet, we are not sure about how we are going to combine all of these.

Do you have any recommendations for other frameworks we could use for this purpose or could you give any ideas on how to proceed?


Solution

  • In case someone is still interested in the problem, there's the MediaPipeUnityPlugin which you could use to directly connect Unity and MediaPipe.