androidandroidxfirebase-mlkitfirebase-machine-learning

inputImage.getByteBuffer() is null when using AndroidX Camera with MLKit


This is the code from my Activity class:

    @Override protected void onCreate(Bundle savedInstanceState)
    {
        requestWindowFeature(Window.FEATURE_NO_TITLE);
        super.onCreate(savedInstanceState);

        _cpf = ProcessCameraProvider.getInstance(this);
        _cpf.addListener(new Runnable() {
            @Override
            public void run() {
                try {
                    ProcessCameraProvider cameraProvider = _cpf.get();
                    bindImageAnalysis(cameraProvider);
                } catch (ExecutionException | InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }, ContextCompat.getMainExecutor(this));
    }

    private void bindImageAnalysis(@NonNull ProcessCameraProvider cameraProvider) {
        ImageAnalysis imageAnalysis =
                new ImageAnalysis.Builder().setTargetResolution(new Size(640, 360))
                        .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST).build();
        imageAnalysis.setAnalyzer(ContextCompat.getMainExecutor(this), new ImageAnalysis.Analyzer() {
            @Override
            public void analyze(@NonNull ImageProxy ip) {
                processImage(ip.getImage(), ip.getImageInfo().getRotationDegrees());
                ip.close();
            }
        });
        CameraSelector cameraSelector = new CameraSelector.Builder().requireLensFacing(CameraSelector.LENS_FACING_BACK).build();
        cameraProvider.bindToLifecycle((LifecycleOwner)this, cameraSelector, imageAnalysis);
    }

On every analyze callback, MLKit indicates that InputImage.fromMediaImage(ip.getImage(),ip.getImageInfo().getRotationDegrees()).getByteBuffer() is null. How should I interpret this? Does this always mean that the image is null?


Solution

  • The problem was in the fact that I was .close()ng prematurely. PoseDetector processing is asynchronous so processImage is being called and immediately after ip.close(). When ip.close() is called, processImage is still working so this is the reason behind Attempt to invoke virtual method 'java.nio.ByteBuffer android.media.Image$Plane.getBuffer(). Instead of using .close() inside the analyze method, it should be invoked at the end of OnSuccessListener or OnSuccessListener.

    Here is the full example.