androidencodeandroid-mediacodecmediamuxer

Android MediaMuxer creates corrupted videos on some devices


I am currently using media muxer with media codec to create and save a video. It works perfectly on many devices, however, some devices have a problem: After releasing the muxer, video cannot be played or restored. MainFrameMuxer.kt

class MainFrameMuxer(path: String, fps: Float) : FrameMuxer {

private val frameUsec: Long = (TimeUnit.SECONDS.toMicros(1L) / fps).toLong()

private val muxer: MediaMuxer = MediaMuxer(path, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4)

var started = false
private var videoTrackIndex = 0
private var videoFrames = 0
private var finalVideoTime: Long = 0

override fun start(videoFormat: MediaFormat) {
    Log.d("mpeg", "starting")
    videoTrackIndex = muxer.addTrack(videoFormat)
    Log.d("Video format", videoFormat.toString())
    muxer.start()
    started = true
}

override fun muxVideoFrame(byteBuffer: ByteBuffer, bufferInfo: MediaCodec.BufferInfo) {
    finalVideoTime = frameUsec * videoFrames++
    bufferInfo.presentationTimeUs = finalVideoTime

    muxer.writeSampleData(videoTrackIndex, byteBuffer, bufferInfo)
}

override fun release() {
    Log.d("mpeg", "release is called")
    muxer.stop()
    muxer.release()
}

override fun getVideoTime(): Long {
    return finalVideoTime
}

}

Video encoding - codec drain: The encoding is made with surface on avc codecs.

private fun alternativeDrainer(endOfStream: Boolean) {
    val encoder = mediaCodec ?: return
    val timeoutUs = TIMEOUT_USEC.toLong()

    if (endOfStream) {
        Log.d("mpeg", "end of stream received successful")
        encoder.signalEndOfInputStream()
    }

    while (true) {
        val outBufferId = encoder.dequeueOutputBuffer(bufferInfo, timeoutUs)

        if (outBufferId >= 0) {
            val encodedBuffer = encoder.getOutputBuffer(outBufferId) ?: continue
            Log.d("v_t", "eb = ${bufferInfo.size}, fl = ${bufferInfo.flags}")

            frameMuxer.muxVideoFrame(encodedBuffer, bufferInfo)

            encoder.releaseOutputBuffer(outBufferId, false)

            if ((bufferInfo.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0)
                break
        } else if (outBufferId == MediaCodec.INFO_TRY_AGAIN_LATER) {
            if (!endOfStream)
                break

        } else if (outBufferId == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
            frameMuxer.start(encoder.outputFormat)
        }
    }
}

Normal working muxer logs with "mpeg" tags:

screenshot

Problematic case on emulator device:

screenshot

The corrupted video has normal size and some restore tools can get its right duration, however it can't be opened at all.

The encoding was tested on xiaomi devices android 9, 10, 11, on some devices with android 5.1 and 10, Samsung android 9, 10, 12. However it works with this corrupter videos issue on google pixel 6 (also on emulators with android 12) and Samsung fold 3.

What could be the problem?


Solution

  • As I've understood properly, the problem was in muxer initialization. On some devices initialization happens after start().

    As a workaround I've put a delay there and a problem was solved.

    override fun start(videoFormat: MediaFormat) {
        runBlocking {
            delay(1000)
            Log.d("mpeg", "starting")
            videoTrackIndex = muxer.addTrack(videoFormat)
            Log.d("Video format", videoFormat.toString())
            muxer.start()
            started = true
        }
    }
    

    Also I've changed a bit drainer function:

    private fun alternativeDrainer(endOfStream: Boolean) {
        val encoder = mediaCodec ?: return
        val timeoutUs = TIMEOUT_USEC.toLong()
    
        if (endOfStream) {
            Log.d("mpeg", "end of stream received successful")
            encoder.signalEndOfInputStream()
        }
    
        while (true) {
            val outBufferId = encoder.dequeueOutputBuffer(bufferInfo, timeoutUs)
    
    
            if (outBufferId >= 0) {
                val encodedBuffer = encoder.getOutputBuffer(outBufferId) ?: continue
                Log.d("v_f", "eb = ${bufferInfo.size}, fl = ${bufferInfo.flags}")
    
                if (!mainMuxer.started) break
                mainMuxer.muxVideoFrame(encodedBuffer, bufferInfo)
                encoder.releaseOutputBuffer(outBufferId, false)
    
                if ((bufferInfo.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0)
                    break
            } else if (outBufferId == MediaCodec.INFO_TRY_AGAIN_LATER) {
                break
            } else if (outBufferId == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
                if (mainMuxer.started) break
                mainMuxer.start(encoder.outputFormat)
            }
        }
    }
    

    After these changes, everything works as expected.