androidglsurfaceviewmediapipe

mediapipe.components.ExternalTextureConverter.setSurfaceTextureAndAttachToGLContext returns null. makes the app crash onResume


I am working on jiuqiant's mediapipe face detection app. I cloned the repo, the app works fine, detects my face but when I pause the app and resumes back, it crashes. with error on this line permalink

converter.setSurfaceTextureAndAttachToGLContext(
previewFrameTexture,
displaySize.getWidth(), 
displaySize.getHeight());

with this log

D/CameraXPreviewHelper: viewSize or frameSize is null.
E/SurfaceView: Exception configuring surface
    java.lang.NullPointerException: Attempt to invoke virtual method 'int android.util.Size.getWidth()' on a null object reference
        at com.example.myfacedetectionapp.MainActivity$1.surfaceChanged(MainActivity.java:146)
        at android.view.SurfaceView.updateSurface(SurfaceView.java:802)
        at android.view.SurfaceView$2.onPreDraw(SurfaceView.java:163)
        at android.view.ViewTreeObserver.dispatchOnPreDraw(ViewTreeObserver.java:977)
        at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:3044)
        at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:1845)
        at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:8235)
        at android.view.Choreographer$CallbackRecord.run(Choreographer.java:994)
        at android.view.Choreographer.doCallbacks(Choreographer.java:806)
        at android.view.Choreographer.doFrame(Choreographer.java:738)
        at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:980)
        at android.os.Handler.handleCallback(Handler.java:873)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:226)
        at android.app.ActivityThread.main(ActivityThread.java:7178)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:503)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:942)

It is clear that setSurfaceTextureAndAttachToGLContext is returning null value. Looking at source code (below is the setSurfaceTextureAndAttachToGLContext function in com.google.mediapipe.components.ExternalTextureConverter)

if (viewSize != null && this.frameSize != null) {
    Size optimalSize = this.getOptimalViewSize(viewSize);
    return optimalSize != null ? optimalSize : this.frameSize;
} else {
    Log.d("CameraXPreviewHelper", "viewSize or frameSize is null.");
    return null;
}

it seems that this.frameSize is null. But I dont know how to fix this.

I also tried to hardcode the values

converter.setSurfaceTextureAndAttachToGLContext(previewFrameTexture,200,200);

but it also gives error

I/GLConsumer: [SurfaceTexture-0-13076-0] attachToContext
E/GLConsumer: [SurfaceTexture-0-13076-0] attachToContext: GLConsumer is already attached to a context
E/BpSurfaceComposerClient: Failed to transact (-1)
E/BpSurfaceComposerClient: Failed to transact (-1)
D/GlThread: Stopping GL thread ExternalTextureConverter
D/CaptureSession: Opening capture session.
E/AndroidRuntime: FATAL EXCEPTION: ExternalTextureConverter
    Process: com.example.myfacedetectionapp, PID: 13076
    java.lang.RuntimeException: Error during attachToGLContext (see logcat for details)
        at android.graphics.SurfaceTexture.attachToGLContext(SurfaceTexture.java:289)
        at com.google.mediapipe.components.ExternalTextureConverter$RenderThread.setSurfaceTextureAndAttachToGLContext(ExternalTextureConverter.java:247)
        at com.google.mediapipe.components.ExternalTextureConverter.lambda$setSurfaceTextureAndAttachToGLContext$2$ExternalTextureConverter(ExternalTextureConverter.java:166)
        at com.google.mediapipe.components.-$$Lambda$ExternalTextureConverter$VQm8xcL00VolxXr4rMIgGUKVKnw.run(Unknown Source:8)
        at android.os.Handler.handleCallback(Handler.java:873)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:226)
        at com.google.mediapipe.glutil.GlThread.run(GlThread.java:141)

Is it some version problem. I cant understand.

steps to recreate -

  1. Clone repo and run.

  2. Thats it. Now try pausing and resuming the app. please let me know if you dont face the error.


Solution

  • Ok using classes given here works. Just place that com directory in your app/src/main.

    Just remember to remove the duplicate classes from .aar file in libs directory.