I want to play audio and video simultaneously using Media Source in Javascript. I can play either but not both at the same time.
Here's my code so far:
let mse = function() {
if ('MediaSource' in window && MediaSource.isTypeSupported(codecs)) {
a = new MediaSource();
a.addEventListener('sourceopen', ma);
return a;
} else {
return false;
}
}();
I have another 600 lines of code, if needed.
if you need to play both of video and sound you need to create 2 source buffers . and as you said . you can play just one of them . so i guess you code is fine . so you need to create 2 buffers . like this
my_media_source.addEventListener("sourceopen",function (){
var video_source_buffer = my_media_source.addSourceBuffer(video_mimeCodec);
var audio_source_buffer = my_media_source.addSourceBuffer(audio_mimeCodec);
//.......
video_source_buffer.appendBuffer(...);
audio_source_buffer.appendBuffer(...);
}
Now you can just buffer both of video and audio , keep in maind that MediaSource will not play you video antil it gets both of data . so for exemple if you buffered the first 5s from video and 3s from audio the player will stop at 3s
just keep your buffer equitable ;)