androidopencv

Is recommended to release CvCameraViewFrame inputFrame?

I am working in an Android app that uses JavaCameraView. So in my activity I implement CvCameraViewListener2.

My implementation of onCameraFrame() looks like this:

@Override
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
    final Mat frame = inputFrame.rgba();
    return bs.process(frame);
}

And my question is: would it be good if I release frame as I am not using it anymore? So to do something like this:

@Override
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
    final Mat frame = inputFrame.rgba();
    Mat result = bs.process(frame);
    frame.release();
    return result;
}

Thanks you for your help.

Solution

  • You don't have to release the inputFrame matrix but you have to be careful with your return matrix.

    Explanation:

    onCameraFrame() callback is called in deliverAndDrawFrame() method from CameraBridgeViewBase:

    protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
    Mat modified;
    if (mListener != null) {
        modified = mListener.onCameraFrame(frame);
    } else {
        modified = frame.rgba();
    }
    ...
}

    deliverAndDrawFrame() method is called in the CameraWorker from JavaCameraView:

    ...
if (!mStopThread && hasFrame) {
    if (!mFrameChain[1 - mChainIdx].empty())
        deliverAndDrawFrame(mCameraFrame[1 - mChainIdx]);
}
...

    mCameraFrame is an array of JavaCameraFrame type. If you look at this class you can see where the matrix comes from:

    @Override
public Mat gray() {
    return mYuvFrameData.submat(0, mHeight, 0, mWidth);
}

@Override
public Mat rgba() {
    Imgproc.cvtColor(mYuvFrameData, mRgba, Imgproc.COLOR_YUV2RGBA_NV21, 4);
    return mRgba;
}

    In case of gray():

    mYuvFrameData is a Mat that is reused with each new frame -> you don't have to release it.

    In case of rgba():

    mYuvFrameData is converted to mRgba. mRgba is a class member initialized in the constructor and reused each time. The cvtColor() method uses create() method to allocate space in the dst matrix. This method works as follow (JavaDoc):

    So, neither have you to release in this case.

    Return matrix

    If we look again the code of deliverAndDrawFrame() method, we can see how our return matrix is processed:

    ...
modified = mListener.onCameraFrame(frame);
...

    Our return matrix is directly assigned, but the previous one is not released.

    So we have two options:

    1. Reuse our return matrix converting it to a class member (recommended).
    2. Modify the CameraBridgeViewBase source code, adding modified.release():

    Option 1 (following your example):

    ...

private Mat processedFrame;

@Override
public void onCameraViewStarted(int width, int height) {
    processedFrame = new Mat();
    ...
}

@Override
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
    final Mat frame = inputFrame.rgba();  
    Mat result = bs.process(frame);   
    result.copyTo(processedFrame);
    result.release(); 
    return processedFrame;
}

    Option 2:

    protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
    ...
    boolean bmpValid = true;
    if (modified != null) {
        try {
            Utils.matToBitmap(modified, mCacheBitmap);
            modified.release(); // Add this line
            ...
}