javajava-threadsproject-loom

Minimum concurrency to benefit from JDK19 virtual threads


According to JEP 425,

virtual threads can significantly improve application throughput when the number of concurrent tasks is high (more than a few thousand), and the workload is not CPU-bound

Why are virtual threads not helpful when the thread count is much lower than a few thousand? If I have 50 concurrent I/O-bound tasks, will I not significantly reduce CPU load by using virtual threads to eliminate heavyweight OS thread context switching?


Solution

  • In short: in case of 50 concurrent tasks you won't gain much by using virtual threads. They won't make things worse either - you'll just hardly notice any difference. But when the number of native threads is high, things start to change.

    How high? A typical application can sustain only a several thousand threads running simultaneously (an empirical observation). Beyond that you'll most likely run out of RAM (thread stacks will take tens of gigabytes of memory), plus constant context switching will slow down your application.

    Virtual threads are switched in user space and their stacks are tiny (hundreds of bytes). Thus, your application will likely survive millions of them.

    So, unless you have a need to big number of clients simultaneously (by big I mean hundreds and thousands), you won't benefit much from using virtual threads. Those are for monstrous I/O bound workloads. But I suggest try and measure, of course.