We are currently working on a Spring Integration flow where each incoming message needs to trigger a call to an external HTTP service. The external call can take up to 20 seconds to complete, and the response can be quite large — it needs to be fully held in memory for further processing.
To improve scalability and reduce the number of blocked platform threads, we want to take advantage of Java’s virtual threads (Project Loom, Java 21+). Our current approach is to use:
Executors.newVirtualThreadPerTaskExecutor()
as the executor for the channel in the integration flow:
IntegrationFlow.from(adapter).channel(c -> c.executor(virtualThreadExecutor)).handle(processingService)
When messages come in, we can already verify in the processingService, that they are processed by a virtual Thread.
Here’s what we’re wondering:
Is this the correct way to use virtual threads in Spring Integration or are there better ways to integrate virtual threads in this kind of flow?
What are the potential pitfalls, limitations, or performance issues we should be aware of when using virtual threads in this context?
Are there specific components in Spring Integration (like gateways, service activators, etc.) that may not yet be virtual-thread-friendly?
To avoid running out of memory, we thought about limiting the number of concurrent external requests. Are there best practises for that?
We’re especially interested in any best practices or real-world experiences anyone might have. Thanks in advance!
We’re currently using SpringBoot 3.4.4 and Java 21.
Is this the correct way to use virtual threads in Spring Integration or are there better ways to integrate virtual threads in this kind of flow?
Yes, that's the way. Whenever you can inject an Executor, that's a candidate for virtual threads.
What are the potential pitfalls, limitations, or performance issues we should be aware of when using virtual threads in this context?
This is more general question about virtual threads nature. Nothing Spring-specific. So, if there is a pinning of the platform thread somewhere down the call, nothing we can from Spring perspective. I think this article does a decent job explaining what problems and where can happen: https://medium.com/@phil_3582/java-virtual-threads-some-early-gotchas-to-look-out-for-f65df1bad0db
Are there specific components in Spring Integration (like gateways, service activators, etc.) that may not yet be virtual-thread-friendly?
All the code in Spring portfolio was optimized to avoid synchronized blocks. We keep our dependencies up-to-date, but that does not mean that some library we rely on is virtual threads friendly. For example, Apache Kafka (even current version 4.0.0) uses a regular Thread for tasks for each consumer instance. See ApplicationEventHandler and how it is used in the AsyncKafkaConsumer.
To avoid running out of memory, we thought about limiting the number of concurrent external requests. Are there best practises for that?
That's probably what we call rate limiting. See more info in docs: https://docs.spring.io/spring-integration/reference/handler-advice/classes.html#rate-limiter-advice