javascriptwebrtcvideo-editingmedia-sourceweb-mediarecorder

Web MediaRecorder API : concatenating video chunks end-pads with blank video


I have working code (below) that concatenates the first and last chunk of video recorded from the browser.

Chunks are collected every three seconds, and the resultant downloaded video does indeed play the first and last chunk in sequence.

However, the total length of the video is the length of all collected chunks, rather than just the sum of the first and last. The "extra" video is just blank.

More to follow, after the code:

let mediaRecorder = new MediaRecorder()
let chunks = []
mediaRecorder.start(3000) // triggers ondataavailable every 3 seconds
mediaRecorder.ondataavailable = (ev) => {
  chunks.push(ev.data);
  if (chunks.length > 2) chunks.splice(1,1) // keeps only the first and last chunk
}
let blob = new Blob(chunks, {'type': 'video/mp4;'}
let videoURL = window.URL.createObjectURL(blob)
let downloadLink = document.getElementById('downloadLink');
downloadLink.href = videoURL;
mediaRecorder.stop();
downloadLink.click()

Maybe ev.timeStamp is being used to determine the video's length, rather than just the sum of the duration of the chunks? That would make more sense if the last chunk were playing back in the last 3 seconds, with blank video in the middle. But no -- it plays [beginning chunk, end chunk, blank space].

Is there a way to prevent the blank video that follows/pads the concatenated beginning/end?

--

Edit: I've also tried slicing off the end of the blob, blob.slice(0, -10,000), but it doesn't trim the blank space.


Solution

  • tl;dr You're skipping some MediaRecorder chunks. You Can't Do Thatâ„¢.

    If this is on Chromium-based or Firefox browsers, MediaRecorder is generating media in a webm / Matroska boxing format. There's MKVToolNix to look at Matroska files; you can do it with a hex dump but that's an over-the-top pain in the xxx neck.

    The a/v material comes in Clusters. Clusters have absolute timestamps (containing times since the beginning of the recording, not since the previous cluster) on them. And, each Cluster contains a bunch of SimpleBlocks holding the media. Although this isn't spelled out in any specs, Chromium emits a video interframe (a/k/a keyframe a/k/a Instantaneous Decoder Refesh frame) into the first video SimpleBlock in the beginning of each Cluster.

    So, if you only save the last chunk, it contains a Cluster with timestamp far in the future from the beginning of the recording. The player waits til that time arrives, then plays it.

    It's a small miracle what you're doing works as well as it does. For stored MediaRecorder output to work predictably you can't skip any chunks, nor can you take them out of order. For one thing, it's a pure implementation detail that the chunks break along Cluster boundaries: nothing in the spec guarantees that. I've seen MediaRecorder break chunks in other places.

    Cutting out a hunk of video timeline from a Matroska / webm stream made by MediaRecorder takes a whole mess of sophisticated parsing and reassembly of the Matroska boxing. And unless you always cut right before interframes, it takes decoding and recoding the video stream too.

    There's the npm ebml package to parse the Matroska data stream.