javascriptaudioweb-audio-api

Web audio echoCancellation with multiple sources does not work

I have an application that plays multiple web audio sources concurrently, and allows the user to record audio at the same time. It works fine if the physical input (e.g. webcam) cannot detect the physical output (e.g. headphones). But if the output can bleed into the input (e.g. using laptop speakers with a webcam), then the recording picks up the other audio sources.

My understanding is the echoCancellation constraint is supposed to address this problem, but it doesn't seem to work when multiple sources are involved.

I've included a simple example to reproduce the issue. JSfiddle seems to be too strictly sandboxed to allow user media otherwise I'd dump it somewhere.

Steps to reproduce

  1. Press record
  2. Make a noise, or just observe. The "metronome" should beep 5 times
  3. After 2 seconds, the <audio> element source will be set to the recorded audio data
  4. Play the <audio> element - you will hear the "metronome" beep. Ideally, the metronome beep would be "cancelled" via the echoCancellation constraint which is set on the MediaStream, but it doesn't work this way.

index.html

<!DOCTYPE html>
<html lang="en">
  <body>
    <button onclick="init()">record</button>
    <audio id="audio" controls="true"></audio>
    <script src="demo.js"></script>
  </body>
</html>

demo.js

let audioContext
let stream
async function init() {
  audioContext = new AudioContext()
  stream = await navigator.mediaDevices.getUserMedia({
    audio: {
      echoCancellation: true,
    },
    video: false,
  })

  playMetronome()

  record()
}

function playMetronome(i = 0) {
  if (i > 4) {
    return
  }

  const osc = new OscillatorNode(audioContext, {
    frequency: 440,
    type: 'sine',
  })
  osc.connect(audioContext.destination)
  osc.start()
  osc.stop(audioContext.currentTime + 0.1)

  setTimeout(() => {
    playMetronome(i + 1)
  }, 500)
}

function record() {
  const recorder = new MediaRecorder(stream)
  const data = []
  recorder.addEventListener('dataavailable', (e) => {
    console.log({ event: 'dataavailable', e })
    data.push(e.data)
  })
  recorder.addEventListener('stop', (e) => {
    console.log({ event: 'stop', e })
    const blob = new Blob(data, { type: 'audio/ogg; codecs=opus' })
    const audioURL = window.URL.createObjectURL(blob)
    document.getElementById('audio').src = audioURL
  })
  recorder.start()
  setTimeout(() => {
    recorder.stop()
  }, 2000)
}
Solution

  • Unfortunately this is a long standing issue in Chrome (and all its derivatives). It should work in Firefox and Safari.

    Here is the ticket: https://bugs.chromium.org/p/chromium/issues/detail?id=687574.

    It basically says that the echo cancellation only works for audio that is coming from a peer connection. As soon as it is processed locally by the Web Audio API it will not be considered anymore by the echo cancellation.