javascriptmedia-source

Javascript Mediasource play video and audio simultaneously?


I want to play audio and video simultaneously using Media Source in Javascript. I can play either but not both at the same time.

Here's my code so far:

 let mse = function() {
    if ('MediaSource' in window && MediaSource.isTypeSupported(codecs)) {
        a = new MediaSource();
        a.addEventListener('sourceopen', ma);
        return a;
    } else {
        return false;
    }
}();

I have another 600 lines of code, if needed.


Solution

  • if you need to play both of video and sound you need to create 2 source buffers . and as you said . you can play just one of them . so i guess you code is fine . so you need to create 2 buffers . like this

     my_media_source.addEventListener("sourceopen",function (){
        var video_source_buffer = my_media_source.addSourceBuffer(video_mimeCodec);
        var audio_source_buffer = my_media_source.addSourceBuffer(audio_mimeCodec);
    
         //.......
    
         video_source_buffer.appendBuffer(...);
         audio_source_buffer.appendBuffer(...);
     }
    

    Now you can just buffer both of video and audio , keep in maind that MediaSource will not play you video antil it gets both of data . so for exemple if you buffered the first 5s from video and 3s from audio the player will stop at 3s

    just keep your buffer equitable ;)