In a highly scalable product, I'm using ScheduledThreadPoolExecutor to send delayed events, one for every incoming message. The delay is around 1 second. Under very high CPU load, the scheduler sends the events at a very low rate. Even though hundreds of events should be sent per second, it barely sends 10, sometimes 1 or 2.
This is the configuration
private final ScheduledThreadPoolExecutor deliveryReceiptExecutorService =
new ScheduledThreadPoolExecutor(50, threadFactory);
My guess is that the CPU is so busy that the threads handling the scheduled events don't get the chance to run.
Question: can somebody recommend a better approach to guarantee at least some balance between the incoming events and the outgoing delayed events? Based on the logs from the scheduler class, after one hour of running the process, completedTaskCount is equal to 300k and taskCount is equal to 3M, 10 times more.
Update: I'm using ECS in AWS to run tasks with 1 vCPU and 4 GB of memory. The idea behind my test is to see what happens when there's a spike of traffic and if something can be done to make sure that the events are sent in real-time, without significant delays.
If your CPU is running at virtually 100% across all cores, and work is not getting done, your computer is overburdened.
Given that you merely want a one second delay at the top of every task, you do not really need a ScheduledExecutorService. Just sleep for a second as the first line of code in your task.
Thread.sleep( Duration.ofSeconds( 1 ) ) ;
Then use virtual threads to run your tasks, for possibly better overall performance.
ExecutorService deliveryReceiptExecutorService = Executors.newVirtualThreadPerTaskExecutor() ;
Caveat: Virtual threads are contraindicated if your tasks are CPU-bound or have long-running sections of code that are either native (Foreign Functions, JNI) or synchronized.