androidandroid-ndkmediapipehand-tracking

mediapipe hand tracking on Android, C++ ndk side


Is it possible to use mediapipe hand landmark detection directly on the NDK side?

The hand landmark solution is in Kotlin, but my project will use GameActivity and I have already a buffer on the C++ side taken from camera2ndk API (yuvN12 format, but easy to convert into RGB(A) if needed). I can see that tasks-vision-0.10.14.aar contains the libmediapipe_tasks_vision_jni.so shared library (and I could take the hand_landmarker.task from the asset folder of the example in https://github.com/google-ai-edge/mediapipe-samples), but I'm puzzled which h files I should use for that shared library and how to code the rest. Which example in C/C++ from mediapipe would be best to start with?


Solution

  • I was able to achieve my goal cross-compiling libhand_landmarker.so for Android. This is found on mediapipe tasks, C api. I identified a small subset (~10) of header files needed and I based the processing function on mediapipe/tasks/c/vision/hand_landmarker/hand_landmarker_test.cc file.