javamultithreadinghtop

Why my java long running Threads (5k+ thread) not utilizing all machines cores (12 cores)?

I've Worte a simple multithread java application, The main method just creates 5k threads, each thread will loop over a list having 5M records to process.

My Machine specs:

My jar is now running, And I use hTop to monitor my application and this is what I can see while its running

enter image description here

And This is how I construct a Thread:

ExecutorService executor = Executors.newCachedThreadPool();
Future<MatchResult> future = executor.submit(() -> {
            Match match = new Match();
            return match.find(this);
        });

Match.class

find(Main main){
// looping over a list of 5M 
// process this values and doing some calculations 
// send the result back to the caller 
// this function has no problem and it just takes a long time to run (~160 min)
}

And now I have some questions:

1- Based on my understanding if I have a multiThreaded process, it'll fully utilize all my cores until the task is completed, so why the work load is only around 0.5 (only half a core is used)?

2- Why my Java app state is "S" (sleeping) while its actually running and filling up the logs file?

3- Why I can only see 2037 threads out of 5k are running (this number was actually less than this and its getting increased over time)

My Target: to utilize all cores and get all this 5k+ done as fast as it can be :)

Solution

  • Thank you guys, I've fixed the problem and now Im having the 12 cores running up to maximum as you see in the picture. :)

    enter image description here

    I actually tried to run this command jstack <Pid> to see the status of my all running threads in this process ID, and I found that 95% of my threads are actually BLOCKED at the logging line, I did some googling and found that I can use AsynchAppender in log4J so logging will not block the thread