javascripthtml5-canvashtml5-videomediastream

Creating MediaStream from canvas and video element


I am creating a MediaStream object and adding a video track to it from a canvas using the captureStream() function. This works fine.

However I am trying to add audio as a separate track from a video element. I cant seem to find a way to get an AudioTrack object from a html video element.

Currently HTMLMediaElement.audioTracks is not supported in Chrome. According to the mozilla developer site I should be able to use HTMLMediaElement.captureStream() to return a MediaStream object from which I should be able to retrieve the separate tracks but I just get 'captureStream is not a function' error.

Perhaps i'm missing something very obvious but I would greatly appreciate any help on this.

Below is my current code:

var stream = new MediaStream();

//Works fine for adding video source    
var videotracks = myCanvas.captureStream().getTracks();
var videostream = videotracks[0];
stream.addTrack(videostream);

//Currently not supported in Chrome
var audiotracks = myVid.audioTracks;
var audiostream = audiotracks[0];
stream.addTrack(audiostream);

Solution

  • To get an audio stream from a video element in a cross-browser way :

    AudioContext API createMediaStreamDestination + createMediaElementSource.

    // if all you need is the audio, then you should even probably load your video in an Audio element
    var vid = document.createElement('video');
    vid.onloadedmetadata = generateAudioStream;
    vid.crossOrigin = 'anonymous';
    vid.src = 'https://dl.dropboxusercontent.com/s/bch2j17v6ny4ako/movie720p.mp4';
    
    function generateAudioStream() {
      var audioCtx = new AudioContext();
      // create a stream from our AudioContext
      var dest = audioCtx.createMediaStreamDestination();
      // connect our video element's output to the stream
      var sourceNode = audioCtx.createMediaElementSource(this);
      sourceNode.connect(dest)
        // start the video
      this.play();
      // your audio stream  
      doSomethingWith(dest.stream)
    }
    
    function doSomethingWith(audioStream) {
      // the audio element that will be shown in the doc
      var output = new Audio();
      output.srcObject = audioStream;
      output.controls = true;
      output.play();
      document.body.appendChild(output);
    }

    To add audio to a canvas stream :

    MediaStream Capture Canvas and Audio Simultaneously