javascriptpythonweb-audio-apiopusweb-mediarecorder

Convert int8 opus-encoded array into raw waveform (Python)


I have a js script that records audio from microphone and sends it via a websocket to a python REST API.

Part of js script (works in console):

const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
var options = {mimeType: "audio/webm;codecs=opus", audioBitsPerSecond:16000};
mediaRecorder = new MediaRecorder(stream, options);
mediaRecorder.addEventListener("dataavailable", function(event) {
    var reader = new FileReader();
    reader.addEventListener("loadend", function() {
        var int8View = new Int8Array(reader.result);
        console.log(int8View);
    });
    reader.readAsArrayBuffer(event.data);
    
});
mediaRecorder.start(200);

If I understand correctly the int8array that this code outputs is opus-encoded audio.

How can I decode it into a raw waveform (something that soundfile.read would return, for example) using python?

I can use ffmpeg to decode opus, but it works on files, how can I turn this array into a proper .opus or .ogg file on python side (since I don't want to send files via a websocket)?


Solution

  • how can I turn this array into a proper .opus or .ogg file?

    You're capturing a Webm file stream via audio/webm;codecs=opus, so that would need to be converted from Webm to Ogg Opus. Instead, you could simply download or send the Webm Opus stream as a file as-is:

    const recordedBuffers = []
    
    reader.addEventListener('loadend', () => {
      recordedBuffers.push(reader.result)
    })
    
    function download() {
      const a = document.createElement('a')
      a.href = URL.createObjectURL(new Blob(recordedBuffers))
      a.download = 'my-recorded-audio.webm'
      a.click()
    }
    
    function saveToServer(url) {
      return fetch(url, {
        method: 'post',
        body: new Blob(recordedBuffers),
        headers: {
          'content-type': 'application/octet-stream',  // optional
          'content-type': 'audio/webm',                // optional
        },
      })
    }