angularweb-audio-apiwebassemblyaudio-worklet

Best way to call wasm module functions in AudioWorkletProcessor?


I am using Angular for development of a video conferencing web application. I am trying to call some wasm functions in my AudioWorkletProcessor class. However, I am quite stuck on how to pass the wasm module into the AudioWorkletProcessor and then call those functions. I was able to instantiate the wasm module from the js file generated from emcc (using command: emcc test.cpp -s WASM=1 -s MODULARIZE=1) and then tried passing this module through the options.processorOptions parameter when we construct the AudioWorkletNode; however, it seems like Function objects cannot be passed as it gives a DataCloneError. Is there a better way of instantiating the wasm module and passing it into the worklet processor so that I can use its functions?

Code of audio.service.ts:

import { Injectable } from '@angular/core';
import { AudioContext, AudioWorkletNode } from 'standardized-audio-context';
import * as Module from '../../../assets/worklet/a.out.js'; // a.out.js is file generated by emcc
declare const WebAssembly: any;

@Injectable()
export class AudioService {
  audioCtx = new AudioContext();

  constructor() {}

  async createNewSetting(track) {
    const srcNode = this.audioCtx.createMediaStreamTrackSource(track);
    const destNode = this.audioCtx.createMediaStreamDestination();

    await this.audioCtx.resume();
    await this.audioCtx.audioWorklet.addModule(
      './assets/worklet/spatial-processor.js'
    );

    // Instantiate the wasm module and put in path to file
    const wasm = await Module({ locateFile: function(s) { return 'assets/worklet/' + s; }});
    
    // Will throw DataCloneError
    const spatialNode = new AudioWorkletNode(
      this.audioCtx,
      'spatial-processor',
      { processorOptions: {
        compiledModule: wasm
      }}
    );

    // Connect the nodes
    srcNode.connect(spatialNode);
    spatialNode.connect(destNode);

    // Return the updated audio stream
    return destNode.stream.getTracks()[0];

  }
}
    

Solution

  • Passing an instantiated WASM module to a worker (or an AudioWorklet) isn't possible. It only works when the WASM module is compiled but not yet instantiated. There is an example on MDN which shows how to do that with a regular worker but it works the same way when using the processorOptions of an AudioWorkletProcessor.

    https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/WebAssembly/Module#sending_a_compiled_module_to_a_worker

    The idea is to use compileStreaming() on the main thread and instantiate() in the worker.

    On the other hand it's also possible to import everything directly into an AudioWorklet as shown in this example. However, I would not recommend doing that since compiling the code on the audio thread may result in an audible glitch.

    https://googlechromelabs.github.io/web-audio-samples/audio-worklet/design-pattern/wasm/