Was trying to execute concurrent operations with Gpars.
Gpars.withPool(6) {
someList.eachParallel {
println "${Thread.currentThread}"
}
}
Initially it seems to work
Thread[ForkJoinPool-1-worker-1,5,main]
Thread[ForkJoinPool-1-worker-6,5,main]
Thread[ForkJoinPool-1-worker-2,5,main]
Thread[ForkJoinPool-1-worker-5,5,main]
Thread[ForkJoinPool-1-worker-3,5,main]
Thread[ForkJoinPool-1-worker-4,5,main]
Thread[ForkJoinPool-1-worker-1,5,main]
Thread[ForkJoinPool-1-worker-6,5,main]
Thread[ForkJoinPool-1-worker-2,5,main]
But after iterating for sometime, some of the threads stops executing.
Thread[ForkJoinPool-1-worker-2,5,main]
Thread[ForkJoinPool-1-worker-3,5,main]
Thread[ForkJoinPool-1-worker-2,5,main]
Thread[ForkJoinPool-1-worker-3,5,main]
Thread[ForkJoinPool-1-worker-2,5,main]
Thread[ForkJoinPool-1-worker-3,5,main]
Thread[ForkJoinPool-1-worker-2,5,main]
Until eventually we are left with only one.
Thread[ForkJoinPool-1-worker-2,5,main]
Thread[ForkJoinPool-1-worker-2,5,main]
Thread[ForkJoinPool-1-worker-2,5,main]
Thread[ForkJoinPool-1-worker-2,5,main]
Thread[ForkJoinPool-1-worker-2,5,main]
Thread[ForkJoinPool-1-worker-2,5,main]
Any idea why this happens? Any solution to keep all threads active?
In my case, it might be worth mentioning that we are iterating at around 50k-200k.
Just in case other people have the same issue. Their documentation says:
Because GParsPool uses a Fork/Join pool (with work stealing), threads may not be applied to a waiting processing task even though they may appear idle. With a work-stealing algorithm, worker threads that run out of things to do can steal tasks from other threads that are still busy.
if you use GParsExecutorsPool , which doesn't use Fork/Join, you get the thread allocation behavior that you would naively expect.
Base on that, I opt to use GParsExecutorsPool instead. With this, all threads executed consistently until the end of the whole eachParallel process.
GParsExecutorsPool.withPool(6) {
someList.eachParallel {
println "${Thread.currentThread}"
}
}