I have two question regarding Zmq:-
1) I am implementing a server which would receive 10K requests/per minute and then redirect these request towards backend. After getting response from backend it will then send response back. I can't use request/reply as restriction in this pattern is recv/reply should be synchronous. Can anyone suggest me as what to use in this scenario.
2) Do I need to implement multi-threading in this also.
Thanks in advance.
Let me answer first the Q1-requirement on performance & Q2-if there is a need for multithreading:
From trivial performance tests, intentionally adding a TCP-transport wrapping overhead, you may taste the "cost" of ZeroMQ message handling layer in [usec] per a message sent.
21.936 [usec]/MSG
111.280 [usec]/MSG
39.714 [usec]/MSG
37.080 [usec]/MSG
11.351 [usec]/MSG
To have an idea about a message batch size impact relief, let's read retest with growing size
>>> [ sender( nBatchSIZE = x ) for x in ( 1, 10, 100, 1000, 10000, 100000, 1000000 ) ]
sent 1 ... RUN took: 58 [usec] i.e. 58.0 [usec]/MSG
sent 10 ... RUN took: 156 [usec] i.e. 15.6 [usec]/MSG
sent 100 ... RUN took: 1071 [usec] i.e. 10.7 [usec]/MSG
sent 1000 ... RUN took: 10561 [usec] i.e. 10.5 [usec]/MSG
sent 10000 ... RUN took: 106478 [usec] i.e. 10.6 [usec]/MSG
sent 100000 ... RUN took: 1333242 [usec] i.e. 13.3 [usec]/MSG
Based on these figures, your Q1 the system will have no problem with 10k/min stream of events on reasonable sized MSGs ( not mentioning multi-GB BLOBs )
The processing performance is thus limited by the back-end phase
Suggestion on Q2-what to use:
The REQ/REP thus goes out of the table. Rather than multi-threading thus try to consider any performance scale-able approaches to increase your end-to-end processing capability -- using more advanced schemes for load-balancers alike REQ/ROUTER||ROUTER/REQ et al, you may both increase the processing speed and add some failure resolution features. Check sample trivialised structures in Single- & Multi-Cluster Architectures.