.netasynchronous.net-corethreadpooliocp

How do Completion Port Threads of the Thread Pool behave during async I/O in .NET / .NET Core?


The .NET / .NET Core Thread Pool uses two different categories of threads internally: worker threads and I/O Completion Port (IOCP) threads. Both are just usual managed threads, but used for different purposes. Via different APIs (e.g. Task.Start or ThreadPool.QueueUserWorkItem) I can start CPU-bound async operations on the worker threads (which shouldn't block, otherwise the Thread Pool would probably create additional worker threads).

But what about performing I/O-bound asynchronous operations? How do the IOCP threads behave exactly in these situations? Specifically, I have the following questions:


Solution

  • Damien and Hans pointed me into the right direction in the comments, which I want to sum up in this answer.

    Damien pointed to Stephen Cleary's awesome blog post which answers the first three points:

    Hans pointed out that there are similar mechanisms to IOCP in Linux (epoll) and MacOS (kqueue).

    UPDATE 2023-04-07: my original explanation of IOCP threads not blocking is wrong. When an IOCP thread is created, it is bound to an I/O Completion Port (on Windows) and then calls GetQueuedCompletionStatusEx in a loop. This call blocks until it is woken up because at least one new event is available on the IOCP. The IOCP thread makes its loop run to process all dequeued events (if you use the TPL, the corresponding task will be updated and the continuation will be queued, either on the caller's SynchronizationContext or on a Thread Pool worker thread). After that, the IOCP will call GetQueuedCompletionStatusEx and either block because no events are available, or the loop body can run again. This can be seen here: ThreadPoolPortable.IO.Windows.cs

    While IOCP threads do block, the following statements are also true:

    UPDATE 2022-06-23: some people asked why IOCP threads do not block during I/O operations. It's important to understand how the Thread Pool manages its threads internally. The Thread Pool keeps a number of threads available, i.e. they reside in memory but are actually in a sleep state. This way when work comes in, you do not pay the cost of creating a new thread (my measurements on the topic show that creating a new thread instead of using an existing one is about 80 times slower). When work is available, it is queued to one of the sleeping threads, their state is changed from sleeping to ready to execute, and thus the operating system can pick this thread up in the next context switch (which usually occurs every 15ms) and assign it to one of your CPU cores. After your work is done, the thread is either put to sleep again or the next task can be executed on it. This is true for both worker threads and IOCP threads.

    To conclude, IOCP threads do not block during the I/O operation, because only once the I/O completion port signals that the operation is complete, work is queued on an IOCP thread to mark the corresponding Task or Task<T> as completed and enqueue a possible continuation either on a worker thread or on the original caller thread if it has a synchronization context assigned to it and ConfigureAwait(false) was not called. During the I/O operation, the IOCP thread that will later execute the aforementioned work is either sleeping or handling completions from other I/O completion ports.