javaandroidkotlincameraandroid-camera2

Camera2: Make SurfaceView aspect fit to fill (scaleType CENTER_CROP)

I've been following the Camera2 example (android/camera-samples/Camera2Video) to create an abstraction over the Camera2 library. The idea of my abstraction is to give the user the ability to use a Camera view from React Native, so it's just a <Camera> view which can be whatever size/aspect ratio.

While this works out of the box on iOS, I can't seem to get the preview on Android to display "what the camera sees" in the correct aspect ratio.

The official example from Android works like this:

If you read the source code of their getPreviewOutputSize(...) function, you might notice that this uses a Display to find the best matching preview size. If I understood the code correctly, this would only work if the camera preview (AutoFitSurfaceView) is exactly the same size as the device's screen. This is poorly designed, as there are lots of cases where that simply isn't true. In my case, the Camera has a bit of a bottom spacing/margin, so it doesn't fill the screen and therefore has weird resolutions (1080x1585 on a 1080x1920 screen)

With that long introduction, here comes my question: How do I actually perform a correct center crop transform (aka scaleType = CENTER_CROP) on my SurfaceView? I've tried the following:

Any help appreciated!

Solution

  • A SurfaceView is a View that also holds a Surface. The Surface is the buffer that the camera needs to output the captured images. The 'View` is what makes it shown in the screen. See the section about SurfaceTexture in Android's Graphic architecture (actually, the whole collection of articles is interesting).

    Introduction To control the aspect ratio of the displayed camera preview, you have to consider two factors:

    Out of scope In the examples below:

    Display the camera preview with correct aspect ratio

    1. Display the SurfaceView object, and call a method to start the capture session as soon as the surface is available. As it is not a @Composable object, you need to embed it into an AndroidView so as to obtain the Context, and use it to create a new instance of SurfaceView. The code below will already display the preview, although not with the correct ratio.
    @Composable
fun ShowCamera(
    cameraDescription: SelectedCameraDescription,
    activateCamera: (Surface) -> Unit,
    modifier: Modifier = Modifier) {

    AndroidView(
        modifier = modifier,
        factory = { context ->
            SurfaceView(context).apply {
                this.post {
                    this.setBackgroundColor(Color.TRANSPARENT)
                    activateCamera(this.holder.surface)
                }
            }
        }
    )
}
    1. Let's extend the surface view into a CameraPreviewSurfaceView so we can have a convenient place to include the code to scale the camera images. The custom view has two tasks. First is to set the size of the surface to the exact size of the resolution we want the camera session to select. This is done in the init method. Second is to scale the surface so it shows on the view with the correct aspect ratio. This is done in the onMeasure method. This method is invoked by the View whenever it needs to adapt itself to the surrounding components. Typically when first inflating the view, or when rotating the device.
    /**
 * A custom [SurfaceView], adapted for camera preview.
 */
@SuppressLint("ViewConstructor") // No need of view constructor, as we're using jetpack.
class CameraPreviewSurfaceView(
    private val cameraDescription: SelectedCameraDescription,
    context: Context): SurfaceView(context) {

    /**
     * Sets the size of the inner surface.
     * The camera session picks the camera resolution from the size of the surface, so it
     * is important to match one of the available resolutions.
     */
    init {
        this.holder.setFixedSize(
            cameraDescription.sizeInPixels.width,
            cameraDescription.sizeInPixels.height)
    }

    /**
     * Recalculates the scale required for the camera output to be displayed on the
     * view without distortion.
     */
    override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
        // Don't interfere with the normal measure procedure:
        super.onMeasure(widthMeasureSpec, heightMeasureSpec)

        // Difference between camera orientation and device orientation:
        val cameraOrientation = cameraDescription.orientation
        val deviceOrientation = deviceOrientation()
        val rotation = cameraOrientation - deviceOrientation

        // Depending on the rotation, the camera width and height are inverted:
        val cameraOutputWidth: Int
        val cameraOutputHeight: Int
        if (rotation % 180 == 0) {
            cameraOutputWidth = cameraDescription.sizeInPixels.width
            cameraOutputHeight = cameraDescription.sizeInPixels.height
        } else {
            cameraOutputWidth = cameraDescription.sizeInPixels.height
            cameraOutputHeight = cameraDescription.sizeInPixels.width
        }

        // Camara output aspect and measured view aspect:
        val cameraOutputAspect =
            cameraOutputWidth.toFloat() / cameraOutputHeight.toFloat()
        val measuredAspect =
            measuredWidth.toFloat() / measuredHeight.toFloat()

        // Surface will be stretched to completely fit the measured area.
        // We want the scale to do the inverse:
        val scaleAspect = cameraOutputAspect / measuredAspect

        if (scaleAspect < 1) {
            scaleX = scaleAspect
            scaleY = 1f
        } else {
            scaleX = 1f
            scaleY = 1f / scaleAspect
        }
    }

    /**
     * Obtains the current orientation of the context,
     * and maps it degrees.
     */
    private fun deviceOrientation(): Int {
        val displayRotation = ContextCompat.getDisplayOrDefault(context).rotation
        return when(displayRotation) {
            Surface.ROTATION_0 -> 0
            Surface.ROTATION_90 -> 90
            Surface.ROTATION_180 -> 180
            Surface.ROTATION_270 -> 270
            else -> 0
        }
    }
}
    1. Call the custom surface view from the user interface:
    @Composable
fun ShowCamera(
    cameraDescription: SelectedCameraDescription,
    activateCamera: (Surface) -> Unit,
    modifier: Modifier = Modifier) {

    AndroidView(
        modifier = modifier,
        factory = { context ->
            CameraPreviewSurfaceView(cameraDescription, context).apply {
                this.post {
                    this.setBackgroundColor(Color.TRANSPARENT)
                    activateCamera(this.holder.surface)
                }
            }
        }
    )
}

    Some additional points