I’m using Project Reactor for a non-blocking IO data pipeline, and I’m thinking of using Spring Integration as an abstraction layer to manage all of the flows and components in my pipeline. I’m currently implementing a Kafka consumer with Project Reactor and I’m starting to wonder whether the Spring Integration wrap can damage my pipeline in terms of blocking and back-pressure.
I would like to know if is considered safe to use Spring Integration and its classes in a full non blocking application, and if there’s any traps I should note.
In most cases Spring Integration components are stateless, so they are safe to use in multi-threaded (or non-blocking) environment. You may suffer some concerns if you use a specific channel adapter which really does a blocking IO operation, e.g. file writing or JDBC INSERT. In this cases you would need to consider to switch to a different thread for this type of operations to not block your reactive flow.
See respective recipe in Project Reactor: https://projectreactor.io/docs/core/release/reference/#faq.wrap-blocking
For that thread switching purpose Spring Integration provides an ExecutorChannel and QueueChannel implementations. However the FluxMessageChannel will fit in some cases pretty well.
The limited back-pressure does not make sense in Spring Integration solution since everything is considered as a stream, infinite flow of data. Therefore in most cases Spring Integration relies on the natural back-pressure where we don't request more until we process the current.
Please, give it a try and don't hesitate coming back to us with some feedback!