I'm facing a specific problem. I scheduled a task to run every day at some time. The task itself consists of the following:
The problem is the external server limits the amount of requests per minute (rpm) to a very low number, let's say 20. So if I get the sublist for additional calls of size 100, then I can only update a fifth of that and the rest is gonna stay without additional info (considering the overall processing speed of the task). Currently I've implemented a rate limiter which basically pauses the task for a minute (e.g. when the request count hits 20) using Thread.sleep and then continues over and over etc.
This just leaves a whole thread busy depending on the size of original list for about 5-6 minutes. I was wondering If I could somehow change the way I'm calling the external server, like scheduling requests to be called based on the rpm limit.
Any help will be aprreciated.
Edit. I'm programatically scheduling a task using spring's ThreadPoolTaskScheduler. The update method is the task I'm running.
public abstract class AbstractUpdateScheduler {
protected ThreadPoolTaskScheduler taskScheduler;
protected abstract void setTaskScheduler(ThreadPoolTaskScheduler taskScheduler);
public abstract void update();
public abstract String getCron();
@PostConstruct
private void execute() {
taskScheduler.schedule(this::update, new
CronTrigger(getCron()));
}
}
There is nothing wrong with using Thread.sleep for your purposes.
You said:
This just leaves a whole thread busy depending on the size of original list for about 5-6 minutes.
Well, your daily-task thread will be sleeping the rest of the entire day until the next execution. Sleeping during most of the few minutes it takes to run your hundred sub-tasks is a drop in the bucket.
Having a single platform thread sleeping most of the day is not really a problem. A platform thread is expensive to launch, but you will be launching only once. Sleeping a platform thread for long periods such as minutes or hours is efficient, with the host operating system (OS) able to schedule other threads for execution then.
Given that your sub-tasks involve network calls and database interactions, they could qualify for execution in virtual threads. You could process all the domain objects in each batch of twenty simultaneously using an ExecutorService backed by virtual threads. But given your very small scale of about a hundred per day, I would not bother. They might as well execute serially in a single thread.
For your daily task, use any one of:
ScheduledExecutorService in Java SE.In that daily task, after instantiating your collection of domain objects retrieved from the database, capture the start time. Then process each domain object. When done with that batch, check the current time, and sleep for the rest of a minute. Wake, repeat until no more domain objects.
The collection class used to track our domain objects need not be thread-safe. We will process them in a single thread, serially, as we are in no hurry (assuming we are certain that we can process far more domain objects per minute than our remote service allows requests-per-minute).
We can use a Queue implementation as our collection, such as LinkedList, pulling one domain object at a time until none remain. For more info, see Guide to the Java Queue Interface.
package work.basil.example.threading;
import java.time.Duration;
import java.time.Instant;
import java.util.*;
public class RateLimiting
{
public static void main ( String[] args )
{
RateLimiting app = new RateLimiting( );
app.demo( );
System.out.println( "INFO - Demo done at " + Instant.now( ) );
}
private void demo ( )
{
// Our daily task. In real work, we would likely put in the `run` method of a class implementing `Runnable`.
Collection < Person > persons = this.fetchDomainObjectsFromDatabase( );
Queue < Person > queue = new LinkedList <>( persons );
System.out.println( "queue = " + queue );
final int REQUESTS_PER_MINUTE = 20;
while ( !queue.isEmpty( ) )
{
int nthPerson = 0;
Instant start = Instant.now( );
while ( ( nthPerson < REQUESTS_PER_MINUTE ) && ( !queue.isEmpty( ) ) )
{
nthPerson = nthPerson + 1;
Person person = queue.remove( ); // Should always succeed (not throw exception) as we already checked for not-empty.
this.processPerson( person );
}
if ( !queue.isEmpty( ) )
{
Duration duration = Duration.between( Instant.now( ) , start.plus( Duration.ofMinutes( 1 ) ) ); // Determine how much of the one minute remains at this point.
System.out.println( "💤 Sleeping until next minute… duration = " + duration );
try { Thread.sleep( duration ); } catch ( InterruptedException e ) { throw new RuntimeException( e ); }
}
}
}
private Collection < Person > fetchDomainObjectsFromDatabase ( )
{
// Simulate retrieval of domain objects from database.
// Produces a collection of 86 distinct names.
return
Arrays.stream(
"Abigail,Alexandra,Alison,Amanda,Amelia,Amy,Andrea,Angela,Anna,Anne,Audrey,Ava,Bella,Bernadette,Carol,Caroline,Carolyn,Chloe,Claire,Deirdre,Diana,Diane,Donna,Dorothy,Elizabeth,Ella,Emily,Emma,Faith,Felicity,Fiona,Gabrielle,Grace,Hannah,Heather,Irene,Jan,Jane,Jasmine,Jennifer,Jessica,Joan,Joanne,Julia,Karen,Katherine,Kimberly,Kylie,Lauren,Leah,Lillian,Lily,Lisa,Madeleine,Maria,Mary,Megan,Melanie,Michelle,Molly,Natalie,Nicola,Olivia,Penelope,Pippa,Rachel,Rebecca,Rose,Ruth,Sally,Samantha,Sarah,Sonia,Sophie,Stephanie,Sue,Theresa,Tracey,Una,Vanessa,Victoria,Virginia,Wanda,Wendy,Yvonne,Zoe"
.split( "," )
)
.map( ( String name ) -> new Person( UUID.randomUUID( ) , name ) )
.toList( );
}
private void processPerson ( Person person )
{
// Run your logic.
// Record result in database.
// Simulate this work with a Thread.sleep.
System.out.println( "Processing person = " + person + " at " + Instant.now( ) );
try { Thread.sleep( Duration.ofSeconds( 1 ) ); } catch ( InterruptedException e ) { throw new RuntimeException( e ); }
}
record Person( UUID id , String name ) { }
}
This code works. When run:
queue = [Person[id=2abd94b3-7f4c-4e1a-956b-4f2b03356517, name=Abigail], Person[id=f408fdc5-2122-4485-ab74-7863d0bf2eb9, name=Alexandra], Person[id=a7e1205b-edec-466a-b6d6-fbcd7396a011, name=Alison], Person[id=14dcac0d-d51c-4b9e-9be3-34212c22d42e, name=Amanda], Person[id=0871d471-6a89-4fba-b8d7-4cc098468c90, name=Amelia], Person[id=a59614ee-d6d9-487b-ab2b-5bf50c1de139, name=Amy], Person[id=2796a781-fc01-48ab-a6aa-f19f470475f1, name=Andrea], Person[id=72a59b88-db60-4655-b201-d00892f0b3a3, name=Angela], Person[id=1e70459e-5deb-4ae2-9760-7d6bae098723, name=Anna], Person[id=214e9d8d-ee25-44d1-afff-1de193accecf, name=Anne], Person[id=5f8e525b-d6db-439e-aa6f-ca6e2fafc6d0, name=Audrey], Person[id=285cf471-a923-4407-b29e-4e45e73f232b, name=Ava], Person[id=fde7c58b-cce5-4750-9f03-463cd09171ed, name=Bella], Person[id=94ac09f5-a9c9-4b3b-be80-2f149d580e48, name=Bernadette], Person[id=7a396ebb-90db-45a2-a6ee-68cb791f4f3a, name=Carol], Person[id=3b04dcf5-86bc-416c-bc85-7c394cc036c2, name=Caroline], Person[id=da0be5e5-1211-4f0a-81cd-22cd4cd3765d, name=Carolyn], Person[id=89739b35-6465-43b7-bc40-955aa4c2fbfa, name=Chloe], Person[id=0110a410-5e37-4d4e-a830-7de636098f2e, name=Claire], Person[id=c20c70c7-5c4f-4fb8-97c6-b6035bb9e80e, name=Deirdre], Person[id=b3da74b8-2eb3-4007-b5c7-d7c222a185c0, name=Diana], Person[id=978021ef-37bd-4c26-85c7-c14732769926, name=Diane], Person[id=f40283bb-cbcc-4028-9c80-68aeea543b97, name=Donna], Person[id=41eb1f13-f696-4a7e-b9ed-2dd42cfdc95a, name=Dorothy], Person[id=b37c05c8-e141-4164-a14c-3f7c17873adf, name=Elizabeth], Person[id=4ddf6940-f15a-4bd4-8a24-aa867797ecc2, name=Ella], Person[id=57d88971-8a87-412f-859a-0e41c42f7e63, name=Emily], Person[id=55fd4051-dba9-4240-b600-614caa500b7a, name=Emma], Person[id=8b0f84d7-6f5e-4f46-8ee9-2e44748f116c, name=Faith], Person[id=9815a52c-0d24-4c88-8001-f42857297568, name=Felicity], Person[id=1315156f-0f4a-480d-8b1d-90b2bc2b00a4, name=Fiona], Person[id=9ad0d8c4-26a5-4209-820e-2eea7a1d12f2, name=Gabrielle], Person[id=cfd23971-7ea8-4c75-9eb9-96fc1e0a753a, name=Grace], Person[id=9f8d5da8-2957-4442-a9dd-3d3b5002acd8, name=Hannah], Person[id=7b5db86c-2b37-4c21-8dde-96fdccf38767, name=Heather], Person[id=4b679c2d-da1e-47ba-b96a-af866a5a98c4, name=Irene], Person[id=b083bb06-dfb9-4af5-80ca-fe8a5149b1af, name=Jan], Person[id=94fa0163-406b-4833-9498-e1e32db3d48f, name=Jane], Person[id=c4751c2e-6a9d-4be3-906f-dd2c6312c9be, name=Jasmine], Person[id=7b26cfa5-7eda-4b2b-8bdd-fbe08e962344, name=Jennifer], Person[id=6cb7428a-c2a0-4dae-80f2-c0b7841623ed, name=Jessica], Person[id=df32d55b-5b5e-4d5b-af92-3781edc21bcc, name=Joan], Person[id=81f19b9d-2229-4737-b539-caef61af58d4, name=Joanne], Person[id=fa3d8405-8900-4dcb-95f6-398308aa0d09, name=Julia], Person[id=7b789596-3755-48ad-97e7-22c1f3128576, name=Karen], Person[id=f86e0170-f386-4380-af1a-de8c830a4909, name=Katherine], Person[id=01ef3665-02ef-4ff9-84a5-aa383b87cdde, name=Kimberly], Person[id=c9b913e0-7077-4948-a47c-2b0eea997c9c, name=Kylie], Person[id=91bae483-e878-4c2d-8944-852289821ff8, name=Lauren], Person[id=cbe7c1a1-6911-4374-a915-aece18e0d83a, name=Leah], Person[id=1be50aab-c86c-481a-9a16-50aab8077c14, name=Lillian], Person[id=cd213071-74bb-4230-b32d-5d79b4de135d, name=Lily], Person[id=c68f490e-2020-43e5-9f89-010d3749f4d5, name=Lisa], Person[id=5ee64335-6bac-4b9e-9f7a-b6e1cb98a3f7, name=Madeleine], Person[id=43ba0c0a-3867-4b0d-8d20-617499487023, name=Maria], Person[id=ef368f8a-d613-4352-91ee-d38a864610f3, name=Mary], Person[id=772cae4d-5548-48cb-9007-5a99146eb368, name=Megan], Person[id=2d883cdb-1b56-463d-8cb5-b276132857bf, name=Melanie], Person[id=08bc8117-8ee6-4508-b52e-62832f1013d7, name=Michelle], Person[id=ec816887-5b4b-4eff-8455-87faedeabacf, name=Molly], Person[id=08f0f026-23bb-4cc6-92b8-77d3a67f9828, name=Natalie], Person[id=fc0a535e-53ee-41d7-b870-a50010c412c2, name=Nicola], Person[id=9ea3a368-00f2-4fde-85ed-86e1f25cc6e3, name=Olivia], Person[id=3c3bac1f-e4f6-49f3-9e2a-5083cda3e93a, name=Penelope], Person[id=2b6d1deb-ea5b-4e2e-aa60-0e617b5fa545, name=Pippa], Person[id=59d91b9f-5bc0-4f48-969b-482b997ca9e9, name=Rachel], Person[id=e292a0b3-2fb7-4b58-8e6a-c4b817db1406, name=Rebecca], Person[id=6bb49fdb-72cb-472d-8485-f1926e00add2, name=Rose], Person[id=4c2ccfcf-0657-438f-ab81-ca92e1382d9f, name=Ruth], Person[id=65f27ae9-b138-4309-9785-54eaa9f61146, name=Sally], Person[id=a548e13a-0d8e-4be8-b9b4-6b812154b25f, name=Samantha], Person[id=9c1f84d7-c430-4bf7-b0de-ef842f65cf08, name=Sarah], Person[id=7f50c02a-9121-402c-8aa3-10be6ae40ee3, name=Sonia], Person[id=eeb039cf-f23c-4c3c-9f1d-fc62491b6ca1, name=Sophie], Person[id=a00a14cf-48d7-44f9-9f75-f2c114b2682c, name=Stephanie], Person[id=1d7667e3-b733-42a9-963f-8da753568240, name=Sue], Person[id=bdd4037d-57f7-4b2a-bfe9-3f930d14d28b, name=Theresa], Person[id=a7b1af71-c6e1-4b49-936d-7b264105cbfc, name=Tracey], Person[id=7288a804-0184-40ec-a7de-989562a720a1, name=Una], Person[id=b5e61cae-1e6b-4b24-ba08-c7421266cc61, name=Vanessa], Person[id=bb52b99d-923e-4ef2-aba5-34e0c632b311, name=Victoria], Person[id=5e289a65-a9e9-4cb8-a1b8-f84f06878f22, name=Virginia], Person[id=29200823-f72e-4d49-9c35-e6d4beeff989, name=Wanda], Person[id=60d9b5a1-802b-4e7d-8098-bd9a92ee289a, name=Wendy], Person[id=d8d83e5f-c6d1-4175-9665-0ac913af776b, name=Yvonne], Person[id=b180ad21-b004-45c3-9acd-18402f41a960, name=Zoe]]
Processing person = Person[id=2abd94b3-7f4c-4e1a-956b-4f2b03356517, name=Abigail] at 2024-04-28T22:06:32.444050Z
Processing person = Person[id=f408fdc5-2122-4485-ab74-7863d0bf2eb9, name=Alexandra] at 2024-04-28T22:06:33.448366Z
Processing person = Person[id=a7e1205b-edec-466a-b6d6-fbcd7396a011, name=Alison] at 2024-04-28T22:06:34.454235Z
…
Processing person = Person[id=0110a410-5e37-4d4e-a830-7de636098f2e, name=Claire] at 2024-04-28T22:06:50.518077Z
Processing person = Person[id=c20c70c7-5c4f-4fb8-97c6-b6035bb9e80e, name=Deirdre] at 2024-04-28T22:06:51.522513Z
💤 Sleeping until next minute… duration = PT39.915957S
Processing person = Person[id=b3da74b8-2eb3-4007-b5c7-d7c222a185c0, name=Diana] at 2024-04-28T22:07:32.454997Z
…
Processing person = Person[id=c4751c2e-6a9d-4be3-906f-dd2c6312c9be, name=Jasmine] at 2024-04-28T22:07:50.553563Z
Processing person = Person[id=7b26cfa5-7eda-4b2b-8bdd-fbe08e962344, name=Jennifer] at 2024-04-28T22:07:51.558814Z
💤 Sleeping until next minute… duration = PT39.890573S
Processing person = Person[id=6cb7428a-c2a0-4dae-80f2-c0b7841623ed, name=Jessica] at 2024-04-28T22:08:32.458726Z
Processing person = Person[id=df32d55b-5b5e-4d5b-af92-3781edc21bcc, name=Joan] at 2024-04-28T22:08:33.462090Z
…
Processing person = Person[id=08bc8117-8ee6-4508-b52e-62832f1013d7, name=Michelle] at 2024-04-28T22:08:50.540338Z
Processing person = Person[id=ec816887-5b4b-4eff-8455-87faedeabacf, name=Molly] at 2024-04-28T22:08:51.545666Z
💤 Sleeping until next minute… duration = PT39.906484S
Processing person = Person[id=08f0f026-23bb-4cc6-92b8-77d3a67f9828, name=Natalie] at 2024-04-28T22:09:32.464408Z
Processing person = Person[id=fc0a535e-53ee-41d7-b870-a50010c412c2, name=Nicola] at 2024-04-28T22:09:33.470762Z
…
Processing person = Person[id=7288a804-0184-40ec-a7de-989562a720a1, name=Una] at 2024-04-28T22:09:50.565962Z
Processing person = Person[id=b5e61cae-1e6b-4b24-ba08-c7421266cc61, name=Vanessa] at 2024-04-28T22:09:51.571917Z
💤 Sleeping until next minute… duration = PT39.890426S
Processing person = Person[id=bb52b99d-923e-4ef2-aba5-34e0c632b311, name=Victoria] at 2024-04-28T22:10:32.468806Z
Processing person = Person[id=5e289a65-a9e9-4cb8-a1b8-f84f06878f22, name=Virginia] at 2024-04-28T22:10:33.472915Z
Processing person = Person[id=29200823-f72e-4d49-9c35-e6d4beeff989, name=Wanda] at 2024-04-28T22:10:34.475635Z
Processing person = Person[id=60d9b5a1-802b-4e7d-8098-bd9a92ee289a, name=Wendy] at 2024-04-28T22:10:35.477096Z
Processing person = Person[id=d8d83e5f-c6d1-4175-9665-0ac913af776b, name=Yvonne] at 2024-04-28T22:10:36.478467Z
Processing person = Person[id=b180ad21-b004-45c3-9acd-18402f41a960, name=Zoe] at 2024-04-28T22:10:37.484562Z
INFO - Demo done at 2024-04-28T22:10:38.490110Z
But ideally, we should leave such scheduling matters to our queue rather than clutter the main logic of our app.
To learn about the different approaches to rate-limiting, see this post, Implementing Rate Limiting in Java from Scratch — Leaky Bucket and Token Bucket implementation. As Comments indicate, you may find a rate-limiting queue from a third-party such as Google Guava.
java.util.concurrent.DelayQueue<E>As Commented, Java does come with one rate-limiting Queue implementation that might do,DelayQueue.
An unbounded blocking queue of
Delayedelements, in which an element generally becomes eligible for removal when its delay has expired.
The catch with DelayQueue is that its elements must implement Delayed. So you would have to either (a) add that functionality to your domain object, or (b) wrap your domain object in another class providing than functionality. I might prefer using the code above rather than going to that bother.