If my machine has 10 physical threads then how can my JVM create 100s of threads?

I cannot understand how JVM is running 100s of threads on my machine if my machine only has 10 physical threads and JVM threads map to OS threads. Okay, the one answer I thought was time-slicing. But that's not the case. I created a simple spring boot application which GETs response from another server that produces a 3 seconds delay. I ran the code with server.tomcat.threads.max=10 in application.properties and without it. The difference is day and night.

With server.tomcat.threads.max=10 -

>ab -n 1000 -c 100 http://localhost:8080/block
This is ApacheBench, Version 2.3 <$Revision: 1913912 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /block
Document Length:        39 bytes

Concurrency Level:      100
Time taken for tests:   304.367 seconds
Complete requests:      1000
Failed requests:        100
   (Connect: 0, Receive: 0, Length: 100, Exceptions: 0)
Total transferred:      172100 bytes
HTML transferred:       39100 bytes
Requests per second:    3.29 [#/sec] (mean)
Time per request:       30436.732 [ms] (mean)
Time per request:       304.367 [ms] (mean, across all concurrent requests)
Transfer rate:          0.55 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.7      0       7
Processing:  3031 28738 4956.5  30102   30238
Waiting:     3028 28738 4956.8  30101   30238
Total:       3034 28739 4955.9  30102   30239

Percentage of the requests served within a certain time (ms)
  50%  30102
  66%  30112
  75%  30120
  80%  30134
  90%  30174
  95%  30205
  98%  30223
  99%  30230
 100%  30239 (longest request)

Without server.tomcat.threads.max=10 -

ab -n 1000 -c 100 http://localhost:8080/block
This is ApacheBench, Version 2.3 <$Revision: 1913912 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /block
Document Length:        39 bytes

Concurrency Level:      100
Time taken for tests:   33.435 seconds
Complete requests:      1000
Failed requests:        909
   (Connect: 0, Receive: 0, Length: 909, Exceptions: 0)
Total transferred:      173415 bytes
HTML transferred:       40415 bytes
Requests per second:    29.91 [#/sec] (mean)
Time per request:       3343.475 [ms] (mean)
Time per request:       33.435 [ms] (mean, across all concurrent requests)
Transfer rate:          5.07 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   1.0      0      11
Processing:  3002 3023  15.7   3021    3172
Waiting:     3002 3023  15.6   3020    3168
Total:       3002 3024  16.3   3021    3172

Percentage of the requests served within a certain time (ms)
  50%   3021
  66%   3026
  75%   3029
  80%   3031
  90%   3041
  95%   3063
  98%   3075
  99%   3078
 100%   3172 (longest request)

So if JVM threads are not platform threads then where do java virtual threads come into the picture? As I tested with virtual threads enabled also and it gave the same results as platform threads.

-Edit: I know the application is creating 100s of threads because I am logging the threads. I also read that server.tomcat.threads.max value is 200 by default. But then again, how are these 200 threads running significantly faster if my machine has only 10 physical threads?

Solution

This is simplified in many ways but should get you the idea of what's going on.

JVM Threads (assuming non virtual) are OS/platform threads. But they are not CPU cores. You can have more JVM threads than CPU cores.
It's true that having more threads than CPU can degrade performance, but only if these threads are busy working.
Tomcat threads are 1 thread per request and in your case each request calls some other resources, so it's doing IO, not CPU work. You're what we call IO-bound.
So, in your case, having 200 threads doing almost nothing is not an issue. On the other hand, having only 10 threads would mean that even though they are not used (waiting for IO), you could only have 10 request accepted at the same time.
This is where blocking vs. async is something. If your code was using async stuff (instead of blocking), you would not need 200 threads, only 10 threads would deliver same performance (assuming 10 CPU cores) because they would be available to use when "waiting for IO" instead of being blocked.
This is where virtual threads become a thing: they "automatically transform" blocking code into async code.

EDIT: If you use a visual tool like VisualVM to connect to your JVM and observe threads in live ("Monitor threads" tab I believe, not profiling), this will probably clarify this as well in a visual way seeing threads running or not.