I am currently using media muxer with media codec to create and save a video. It works perfectly on many devices, however, some devices have a problem: After releasing the muxer, video cannot be played or restored. MainFrameMuxer.kt
class MainFrameMuxer(path: String, fps: Float) : FrameMuxer {
private val frameUsec: Long = (TimeUnit.SECONDS.toMicros(1L) / fps).toLong()
private val muxer: MediaMuxer = MediaMuxer(path, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4)
var started = false
private var videoTrackIndex = 0
private var videoFrames = 0
private var finalVideoTime: Long = 0
override fun start(videoFormat: MediaFormat) {
Log.d("mpeg", "starting")
videoTrackIndex = muxer.addTrack(videoFormat)
Log.d("Video format", videoFormat.toString())
muxer.start()
started = true
}
override fun muxVideoFrame(byteBuffer: ByteBuffer, bufferInfo: MediaCodec.BufferInfo) {
finalVideoTime = frameUsec * videoFrames++
bufferInfo.presentationTimeUs = finalVideoTime
muxer.writeSampleData(videoTrackIndex, byteBuffer, bufferInfo)
}
override fun release() {
Log.d("mpeg", "release is called")
muxer.stop()
muxer.release()
}
override fun getVideoTime(): Long {
return finalVideoTime
}
}
Video encoding - codec drain: The encoding is made with surface on avc codecs.
private fun alternativeDrainer(endOfStream: Boolean) {
val encoder = mediaCodec ?: return
val timeoutUs = TIMEOUT_USEC.toLong()
if (endOfStream) {
Log.d("mpeg", "end of stream received successful")
encoder.signalEndOfInputStream()
}
while (true) {
val outBufferId = encoder.dequeueOutputBuffer(bufferInfo, timeoutUs)
if (outBufferId >= 0) {
val encodedBuffer = encoder.getOutputBuffer(outBufferId) ?: continue
Log.d("v_t", "eb = ${bufferInfo.size}, fl = ${bufferInfo.flags}")
frameMuxer.muxVideoFrame(encodedBuffer, bufferInfo)
encoder.releaseOutputBuffer(outBufferId, false)
if ((bufferInfo.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0)
break
} else if (outBufferId == MediaCodec.INFO_TRY_AGAIN_LATER) {
if (!endOfStream)
break
} else if (outBufferId == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
frameMuxer.start(encoder.outputFormat)
}
}
}
Normal working muxer logs with "mpeg" tags:
Problematic case on emulator device:
The corrupted video has normal size and some restore tools can get its right duration, however it can't be opened at all.
The encoding was tested on xiaomi devices android 9, 10, 11, on some devices with android 5.1 and 10, Samsung android 9, 10, 12. However it works with this corrupter videos issue on google pixel 6 (also on emulators with android 12) and Samsung fold 3.
What could be the problem?
As I've understood properly, the problem was in muxer initialization. On some devices initialization happens after start().
As a workaround I've put a delay there and a problem was solved.
override fun start(videoFormat: MediaFormat) {
runBlocking {
delay(1000)
Log.d("mpeg", "starting")
videoTrackIndex = muxer.addTrack(videoFormat)
Log.d("Video format", videoFormat.toString())
muxer.start()
started = true
}
}
Also I've changed a bit drainer function:
private fun alternativeDrainer(endOfStream: Boolean) {
val encoder = mediaCodec ?: return
val timeoutUs = TIMEOUT_USEC.toLong()
if (endOfStream) {
Log.d("mpeg", "end of stream received successful")
encoder.signalEndOfInputStream()
}
while (true) {
val outBufferId = encoder.dequeueOutputBuffer(bufferInfo, timeoutUs)
if (outBufferId >= 0) {
val encodedBuffer = encoder.getOutputBuffer(outBufferId) ?: continue
Log.d("v_f", "eb = ${bufferInfo.size}, fl = ${bufferInfo.flags}")
if (!mainMuxer.started) break
mainMuxer.muxVideoFrame(encodedBuffer, bufferInfo)
encoder.releaseOutputBuffer(outBufferId, false)
if ((bufferInfo.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0)
break
} else if (outBufferId == MediaCodec.INFO_TRY_AGAIN_LATER) {
break
} else if (outBufferId == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
if (mainMuxer.started) break
mainMuxer.start(encoder.outputFormat)
}
}
}
After these changes, everything works as expected.