boost::beast::websocket How to handle multi async_write() function call?

I'm not a network programmer, my design choices are just based on my Google searches.

My Scenario

My clients could receive some events reported by other clients from the server consecutively, and asynchronously there are other requests from the client to the server. Also, having ping/pong is required. My flow is something like this per-client:


+--------+                   +--------+ 
|        |                   |        | --------+
| client | -(1) Connect ---> | server | ----+   |
+--------+                   +--------+     |   |
   ^  ^                                     |   |
   |  |       Accepts the connection        |   |     
   +--|--(2)- and responds to "other" ------+   |
      |       requests.                         |
      |                                         |
      |       Also, I want to send the          |
      +--(3)- events from other threads --------+
              using the established 
              connection.

Why from another thread? Because those events are some status from other clients and I'm keeping those data in a shared data structure and I have a watcher thread to send events to other clients.

My Implementation

I used the example from beast/example/websocket/server/async/websocket_server_async.cpp.

My Problem

Is boost's rules: using_websocket/notes.html that I don't understand. When the client's connection gets accepted, I will queue an async read operation, meanwhile, I also want to send the events to the client and if I do it too quickly I will face an abort that is complaining about two active async write operation.

My Question

How should I handle multi-async_write operations? Or which design should I choose for this?

Possible Duplicate Questions:

Question about concurrent reads/writes#1092: Here vinniefalco just said use strand, but I'm already using them, new instance for each session.
boostbeast-multiple-async-write-...: Here Sehe said: "To fix that you'll just need to queue.', but how? The only thing I didn't use was boost::asio::post and boost::asio::bind_executor; nothing changed even when I used them.
- Using boost::asio::bind_executor caused me an error that was fixed by: boost-asio-executor....
- I also read their chats, but wasn't any luck for me.
boost-beast-async-write-with-queue: The Kai Petzke's answer seems to be the solution to my question about "How to queue?". But, I'm still unsure if my design is right.

_{Compiler: Clang 12, Clang 18
C++ version: 17
OS: Linux 5.10.0-1, Linux 6.10.10-arch1-1
Boost: 1.86.0
Arch: AArch64, x86}

Solution

I had the same issue recently. Boost Asio doesn't have the best documentation, or arguably the best design. Normally, with thread-unsafe code, the simple solution is to use a mutex to avoid multiple threads from executing the dangerous code in two threads at the same time. But Asio makes it quite hard to lock the right things at the right time, for the right duration.

My solution is to have a single outbound message queue (non-asio) that's protected by a regular C++11 mutex. Writing threads lock the mutex, and move a string into that queue. The move is fast enough, this doesn't cause a lot of contention.

If this is the first message in the queue, I schedule an async_write. That will writes the first message in the queue. On completion, it removes the first message from the queue. If there's now another message in the queue, the completion handler for the first async_write schedules the second async_write.

This means that every async_write call happens at a moment when there's no outstanding async_write. - either because there are no messages in the queue, or because we're in an async_write completion handler. Any message added to a non-empty queue does not trigger async_write immediately.

In a more mature library, I would expect async_write to handle this internally. It's already a "composed operation" in Boost Asio terms, in that it calls async_write_some repeatedly. It should be trivial to extend that logic to "call async_write_some until done".