androidkotlincodec

Codec2 implementation for Android


I'm looking for Codec2 impl for Android. Found THIS library, looks promising

I've tried to implement encode-decode logic like below, but I'm getting very distorted, hardly understandable audio output. When I try to use built-in decoder then output is unrecognizable at all (but works with sample file from assets). What I'm missing?

companion object {
    public const val codec2mode = Codec2.CODEC2_MODE_3200

    private const val sampleRate = 8000 // enough
    private const val audioFormat = AudioFormat.ENCODING_PCM_16BIT
    private const val monoStereoIn = AudioFormat.CHANNEL_IN_MONO
    private const val monoStereoOut = AudioFormat.CHANNEL_OUT_MONO
}

private val minBufSize = AudioRecord.getMinBufferSize(sampleRate, monoStereoIn, audioFormat)

private val c2instance = Codec2.create(codec2mode)
private val bits = Codec2.getBitsSize(c2instance) // bytes?
private val samples = Codec2.getSamplesPerFrame(c2instance)

fun recordData(): CharArray {
    Timber.i("recordData bits:$bits, samples:$samples, minBufSize:$minBufSize")
    val recorder = getRecorder()

    val recorderBuffer = ShortArray(samples.coerceAtLeast(minBufSize))
    val framesNum = recorderBuffer.size / samples

    Timber.i("startRecording recorderBuffer size:${recorderBuffer.size} framesNum:$framesNum")
    recorder.startRecording()

    val start = System.currentTimeMillis()

    val encodedBuffer = CharArray(bits)
    var encodedBufferSum = CharArray(0)

    var it = 50
    while (it > 0) {
        recorder.read(recorderBuffer, 0, recorderBuffer.size)
        for (i in 0 until framesNum) {
            Codec2.encode(c2instance, recorderBuffer, encodedBuffer)
            encodedBufferSum += encodedBuffer
        }
        it--
    }

    Timber.i(
        "encode finished in " + (System.currentTimeMillis() - start) +
            "ms, data length:${encodedBufferSum.size}",
    )

    return encodedBufferSum
}

fun playData(encodedBuffer: CharArray) {
    Timber.i("playData bits:$bits, samples:$samples, minBufSize:$minBufSize")
    val start = System.currentTimeMillis()

    val audioTrack = getAudioTrack().apply { play() }

    var workingCopy = encodedBuffer.clone()
    var it = 0
    while (workingCopy.isNotEmpty()) {
        val frame = workingCopy.slice(0 until bits).toCharArray()

        var codec2Buffer = ByteArray(0)
        frame.forEach { codec2Buffer += it.code.toByte() }

        val playbackAudioBuffer = ShortArray(samples)
        Codec2.decode(c2instance, playbackAudioBuffer, codec2Buffer)

        audioTrack.write(playbackAudioBuffer, 0, playbackAudioBuffer.size)

        workingCopy = workingCopy.slice(bits until workingCopy.size).toCharArray()

        it++
    }

    Timber.i(
        "decode and play finished in " + (System.currentTimeMillis() - start) +
            "ms, iterations:$it",
    )
}

output log:

  recordData bits:8, samples:160, minBufSize:640
  startRecording recorderBuffer size:640 framesNum:4
  encode finished in 4138ms, data length:1600
  playData bits:8, samples:160, minBufSize:640
  decode and play finished in 3999ms, iterations:200

btw. I've found THIS project implementing above lib, probably works, but I don't have devices to try out... and HERE we have already built AARs (personally using 0.8-SNAPSHOT64-2)


Solution

  • answer current for version 1.2.0

    bug is burried on JNI side in encode method, in there happens some unnecessary dividing/downsampling. instead of

    short v = (short) jbuf[i * 2];
    

    should be

    short v = (short) jbuf[i];
    

    for current version there is a simple workaround - just add some extra dummy bytes (every second/even position), which are going to be ommited by buggy for loop

    val frameBuffer: ShortArray = ... e.g. from AudioRecorder
    var bugAround = ShortArray(0)
    frameBuffer.forEach {
        bugAround += it // one short  single sample
        bugAround += 0x00
    }
    // bugAround now have 2x size as frameBuffer
    Codec2.encode(c2instance, bugAround/*frameBuffer */, encodedBuffer)
    

    more desription and workaround in issue on github