c++windowsnetwork-programmingtcpiocp

Does WSASend guarantee processing order in IOCP model on Windows?

I'm developing a server using the IOCP (I/O Completion Port) model on Windows, and I have questions about the ordering guarantees of WSASend operations. Specifically, I need to understand if Windows processes WSASend operations in the same order they were called, independent of TCP's ordered delivery guarantee. For example, if I make two consecutive WSASend calls, and the first call has a larger data payload that takes longer to copy, I'm uncertain about how Windows handles this scenario. I need to understand both the sending order on the server side and the receiving order on the client side when using WSARecv with GetQueuedCompletionStatus.

I'm implementing an IOCP server where the order of message processing is critical. While I understand that TCP guarantees ordered delivery of packets, I'm specifically concerned about the behavior of the Windows IOCP implementation itself.

I want to know:

  1. If WSASend operations are processed in the exact order they are called by the application
  2. Whether the client's WSARecv (when used with GetQueuedCompletionStatus) will receive the data in the same order as the server's WSASend calls
  3. How Windows handles scenarios where the first WSASend call might take longer to process due to larger data size

I've searched through Windows documentation but haven't found clear answers about the ordering guarantees specific to IOCP operations.

Solution

  • WSASend simply adds the buffer to an OS-internal list of buffers to send. That's why you need to ensure that the buffer remains valid until completion. The internal list is of course a FIFO. Adding something to this list is practically instant, and does not depend on the size of the buffer that's added.

    WSARecv when called with multiple buffers will use those buffers in the order indicated, and to capacity. In particular, since TCP doesn't have messages, the first buffer will be filled to the last byte before anything is written to the second buffer. WSARecv can't put the N'th message from WSASend in the N'th receive buffer. (Unless you happen to have fixed-size messages and matching fixed-size buffers, in which case this happens by protocol design)