node.jsexpressnode-worker-threadsnodejs-cluster

How to make worker_threads in an express app deal with request and response object?


I want to make worker_threads in an express app deal with request and response object so that the main thread of the process is free to accept the next incoming request. I know clustering is a way where processes are dealing with the request. But I want to know specifically about the worker_threads if this is possible, so that even one process can accommodate even more requests and work independently with the worker_thread.

I want something like threads must work independently dealing the request and response object (definitely req and res object must not be in transferlist and only be sent a structured clone version so that req and res object is still available to the process) while the express instance is free for accepting next request and again create another worker_thread for that and as many as this operation keeps on going.

Might be multiple threads above the limit of a core supports is useless to create but I do not want to focus that condition right now.


Solution

  • You cannot pass request/response objects directly to worker threads as they're not serializable. check this answer

    Instead, extract the necessary data from the request object and pass that to worker threads:

    // In your Express route
    app.post('/endpoint', (req, res) => {
      const data = {
        // Extract needed data
        body: req.body,
        params: req.params,
        query: req.query
      };
      
      // Pass to worker thread
      worker.postMessage(data);
      
      // Set up message handler
      worker.on('message', (result) => {
        // Send the result from worker back to client
        res.status(200).send(result);
      });
    });
    
    1. Process the extracted data in the worker thread and send results back to the main thread.