I'm trying to mux an array of bitmaps into an MP4 video using MediaCodec Library.
Im mostly using this (https://stackoverflow.com/a/51278232/2041475) implementation, but I noticed that on my Google Pixel 6A Pro, I get really bad results, and I think it's because of the ColorFormat.
Even weirder is that the deprecated CodecCapabilities.COLOR_FormatYUV420Planar
produces a somewhat legible image than the recommended CodecCapabilities.COLOR_FormatYUV420Flexible
.
Deprecated (left) vs. Recommended (right):
I'm not entirely sure what's going on, so I need would like some hepl on how to choose the right colorformat.
CodecCapabilities.COLOR_FormatYUV420Flexible
works fine on my S25 Ultra for some reason
When the MediaCodec
processes an input frame it needs to know how to read it. Since a multitude of formats exists it looks for hints in the MediaFormat
object.
Both of the color formats that you tried are problematic.
COLOR_FormatYUV420Planar
is incorrect. Looking at the implementation mentioned in the question (function encodeYUV420SP
) it appears that the input image has a semi-planar layout. So, you should use COLOR_FormatYUV420SemiPlanar
instead. But, as you know, it's deprecated.
COLOR_FormatYUV420Flexible
is a generic format that can represent any layout (planar, semi-planar, packed). The codec needs additional information to know the layout. This is why, when using COLOR_FormatYUV420Flexible
you should work with input images* instead of input buffers. It means calling mediaCodec.getInputImage(index) instead of getInputBuffer().
[*] An image object combines the pixel data in the form of byte buffers with metadata describing the layout of the color planes.