Gating left channel when right channel is above a certain volume using AudioContext

In my chrome extension, I am using AudioContext and AudioNode to merge two streams into a single stream, where one is on the left channel and the other on the right channel. I'd like to also gate (mute) the right channel when the left channel volume is above a certain threshold before merging for recording purposes. The real world use case for this is that the left stream is a direct audio recording of the audio playing through the current tab and the right stream is the microphone input. If a user is not wearing headphones then the microphone input picks up a (slightly timeshift-delayed) bleed of the stream playing and I want to mute that out. In my use case, the recording will always be a two-way conversation so there is practically no need for both people to be speaking at once, and in that case I would still preferrentially prioritize the left stream.

Below is my code to merge the audio streams. I believe that AudioWorklet can help me process the stream in realtime before recording it, or perhaps using a CustomNode - but am failing to find enough instructive examples using these Audio APIs.

const mergeAudioStreams = (leftStream: MediaStream, rightStream: MediaStream) => {
  // Create an AudioContext
  const audioContext = new AudioContext();

  const leftSource = audioContext.createMediaStreamSource(leftStream);
  const rightSource = audioContext.createMediaStreamSource(rightStream);
  
  const merger = audioContext.createChannelMerger(2);
  merger.channelInterpretation = 'speakers'
  merger.channelInterpretation = 'speakers';
  
  leftSource.connect(merger, 0, 0); // Left channel
  rightSource.connect(merger, 0, 1); // Right channel

  const destination = audioContext.createMediaStreamDestination();
  merger.connect(destination);

  // Create a new MediaStream from the MediaStreamDestination
  const combinedStream = destination.stream;

  return combinedStream;
};

Thanks for any direction.

It sounds like your problem would be solved by setting echoCancellation to true when asking for the microphone input.

navigator.mediaDevices.getUserMedia({ audio: { echoCancellation: true } });

If that doesn't work you could also use an AudioWorklet to switch between the streams.

I set up a demo which uses two oscillators as source nodes. The volume of the left channel gets modulated with a GainNode to simulate your use case.

https://stackblitz.com/edit/vitejs-vite-jypuvr

The process() function inside of the AudioWorklet computes the volume of the left channel and uses that to decide if it should pass on the left or right channel.

process([input], [output]) {
  if (input.length === 2) {
    let volume = 0;

    for (let i = 0; i < input[0].length; i += 1) {
      volume += input[0][i] ** 2;
    }

    volume = Math.sqrt(volume / input[0].length);

    if (volume > 0.4) {
      output[0].set(input[0]);
    } else {
      output[0].set(input[1]);
    }
  }

  return true;
}

The demo only ever looks at the samples of the current render quantum and performs a hard switch. You may need to adjust this to make the result a little more pleasant.