encodermp4boxmuxerwebcodecs

Windows Media Player doesn't reproduce audio from file that was created with mp4box library


I am using the mp4box library to create a media file that includes video and audio tracks. I have set up my audio encoder output callback using the following code:

if (encodingAudioTrack === null) {
    let encodingAudioTrackOptions = {
        timescale: this.timescale, 
        samplerate: this.options.audioConfig?.sampleRate, 
        channel_count: this.options.audioConfig?.numberOfChannels,
        samplesize: 16,
        hdlr: 'soun',
        name: 'SoundHandler',
        type: 'mp4a',
        brands: ['mp42', 'mp41', 'isom'],
    }
    encodingAudioTrack = this.file.addTrack(encodingAudioTrackOptions);
}

if (this.previousEncodedAudioChunk) {
    let ab = new ArrayBuffer(this.previousEncodedAudioChunk.byteLength);
    this.previousEncodedAudioChunk.copyTo(ab);
    const sampleDuration = chunk.timestamp - this.previousEncodedAudioChunk.timestamp;
    this.file.addSample(encodingAudioTrack, ab, {
        dts: this.audioSampleTimestamp,
        cts: this.audioSampleTimestamp,
        duration: sampleDuration,
    }); 
    this.audioChunkCount++;
    this.audioSampleTimestamp = this.audioSampleTimestamp + (sampleDuration ?? 0);
}

This is the config for AudioEncoder from WebCodec API:

audioEncoderConfig = {
    codec: 'mp4a.40.2',
    sampleRate: combinedStream.getAudioTracks()[0].getSettings().sampleRate ?? 0,
    numberOfChannels: combinedStream.getAudioTracks()[0].getSettings().channelCount ?? 0,
    bitrate: 128_000,
}

However, when I save the created file and launch it with Windows 10 Media Player or Movies & TV, the audio doesn't play. MPC_HC Player, on the other hand, opens the file correctly. I noticed that when I check the info with MediaInfo tool, the codec identifier is mp4a instead of mp4a.40.2 that was set.

Am I missing some details or did I set something wrong? I appreciate any help.


Solution

  • When adding audio track, you need to provide description property (esds box)

    mp4File.addTrack({
      ...otherAudioTrackOpts,
      description: createESDSBox(meta.decoderConfig?.description)
    })
    

    Full code


    Unfortunately, there is a bug in mp4box.js creating esds box, refer to https://github.com/gpac/mp4box.js/issues/341.

    Missing data causes the size to be calculated incorrectly, which ultimately prevents the file from being parsed.

    If you want a quick fix, you also need to modify package.json. https://github.com/hughfenghen/WebAV/blob/3936c814333e19497b0dd8f40d6adebb3e334693/packages/av-cliper/package.json#L33

    "dependencies": {
      "mp4box": "hughfenghen/mp4box.js"
    }
    

    This issue contains more detail: https://github.com/hughfenghen/WebAV/issues/10