androidandroid-camera2android-mediacodec

Deprecated ColorFormat produces better result than recommended


I'm trying to mux an array of bitmaps into an MP4 video using MediaCodec Library.

Im mostly using this (https://stackoverflow.com/a/51278232/2041475) implementation, but I noticed that on my Google Pixel 6A Pro, I get really bad results, and I think it's because of the ColorFormat.

Even weirder is that the deprecated CodecCapabilities.COLOR_FormatYUV420Planar produces a somewhat legible image than the recommended CodecCapabilities.COLOR_FormatYUV420Flexible.

Deprecated (left) vs. Recommended (right):

I'm not entirely sure what's going on, so I need would like some hepl on how to choose the right colorformat.

CodecCapabilities.COLOR_FormatYUV420Flexibleworks fine on my S25 Ultra for some reason


Solution

  • When the MediaCodec processes an input frame it needs to know how to read it. Since a multitude of formats exists it looks for hints in the MediaFormat object.

    Both of the color formats that you tried are problematic.

    COLOR_FormatYUV420Planar is incorrect. Looking at the implementation mentioned in the question (function encodeYUV420SP) it appears that the input image has a semi-planar layout. So, you should use COLOR_FormatYUV420SemiPlanar instead. But, as you know, it's deprecated.

    COLOR_FormatYUV420Flexible is a generic format that can represent any layout (planar, semi-planar, packed). The codec needs additional information to know the layout. This is why, when using COLOR_FormatYUV420Flexible you should work with input images* instead of input buffers. It means calling mediaCodec.getInputImage(index) instead of getInputBuffer().

    [*] An image object combines the pixel data in the form of byte buffers with metadata describing the layout of the color planes.