I am trying to understand the threading model of ZeroMQ.
According to their white paper http://zeromq.org/whitepapers:architecture#toc3, each I/O thread that the context is created with maps directly to an OS thread. What I don't understand is why two background threads are spawned by this minimal program:
#include <zmq.hpp>
int main() {
zmq::context_t context{1};
zmq::socket_t socket(context, ZMQ_SUB);
socket.setsockopt(ZMQ_SUBSCRIBE, "", 0);
socket.connect("tcp://localhost:5555");
}
As evidenced by the following gdb output, the line that creates the socket spawns two background threads.
Breakpoint 3, 0x00007f3cbebfbc80 in clone () from /lib64/libc.so.6
(gdb) bt
#0 0x00007f3cbebfbc80 in clone () from /lib64/libc.so.6
#1 0x00007f3cbdff7f7a in do_clone.constprop () from /lib64/libpthread.so.0
#2 0x00007f3cbdff9469 in pthread_create@@GLIBC_2.2.5 () from /lib64/libpthread.so.0
#3 0x00007f3cbf74547d in zmq::thread_t::start(void (*)(void*), void*) () from /lib64/libzmq.so.5
#4 0x00007f3cbf70044a in zmq::ctx_t::start_thread(zmq::thread_t&, void (*)(void*), void*) const () from /lib64/libzmq.so.5
#5 0x00007f3cbf7012c1 in zmq::ctx_t::create_socket(int) () from /lib64/libzmq.so.5
#6 0x0000000000401251 in zmq::socket_t::init (this=0x7ffd5d0323f0, context_=..., type_=2) at /usr/include/zmq.hpp:649
#7 0x00000000004010a1 in zmq::socket_t::socket_t (this=0x7ffd5d0323f0, context_=..., type_=2) at /usr/include/zmq.hpp:463
#8 0x0000000000400e2f in main () at sub.cpp:5
(gdb) c
Continuing.
[New Thread 0x7f3cbc7e6700 (LWP 15499)]
Breakpoint 3, 0x00007f3cbebfbc80 in clone () from /lib64/libc.so.6
(gdb) bt
#0 0x00007f3cbebfbc80 in clone () from /lib64/libc.so.6
#1 0x00007f3cbdff7f7a in do_clone.constprop () from /lib64/libpthread.so.0
#2 0x00007f3cbdff9469 in pthread_create@@GLIBC_2.2.5 () from /lib64/libpthread.so.0
#3 0x00007f3cbf74547d in zmq::thread_t::start(void (*)(void*), void*) () from /lib64/libzmq.so.5
#4 0x00007f3cbf70044a in zmq::ctx_t::start_thread(zmq::thread_t&, void (*)(void*), void*) const () from /lib64/libzmq.so.5
#5 0x00007f3cbf70135f in zmq::ctx_t::create_socket(int) () from /lib64/libzmq.so.5
#6 0x0000000000401251 in zmq::socket_t::init (this=0x7ffd5d0323f0, context_=..., type_=2) at /usr/include/zmq.hpp:649
#7 0x00000000004010a1 in zmq::socket_t::socket_t (this=0x7ffd5d0323f0, context_=..., type_=2) at /usr/include/zmq.hpp:463
#8 0x0000000000400e2f in main () at sub.cpp:5
(gdb) c
Continuing.
[New Thread 0x7f3cbbfe5700 (LWP 15500)]
Breakpoint 4, main () at sub.cpp:6
6 socket.setsockopt(ZMQ_SUBSCRIBE, "", 0);
(gdb) info thread
Id Target Id Frame
3 Thread 0x7f3cbbfe5700 (LWP 15500) "a" 0x00007f3cbebfc2c3 in epoll_wait () from /lib64/libc.so.6
2 Thread 0x7f3cbc7e6700 (LWP 15499) "a" 0x00007f3cbebfc2c3 in epoll_wait () from /lib64/libc.so.6
* 1 Thread 0x7f3cbfb4e840 (LWP 15498) "a" main () at sub.cpp:6
(gdb)
Using ZeroMQ and cppzmq-devel 4.1.4
Make with: g++ -std=c++11 -g sub.cpp -lzmq
why two background threads are spawned by this minimal program ?
ZeroMQ is a broker-less asynchronous messaging infrastructure, built around the Context()
-instance(s). If not familiar with the ZeroMQ way, one may enjoy this 5-second read about the main conceptual elements in ZeroMQ hierarchy in less than a five seconds section.
Given the fact, there is always as chance to define how many IO-threads are to be spawned in a Context( nIOthreads = 1 )
to be instantiated, lets put it short:
Context( 0 )
+ 0
IO-threads, i.e. 1 threadContext( 1 )
+ 1
IO-thread, i.e. 2 threadsContext( N )
+ N
IO-thread, i.e. 1+N threadsgiven the experiment above, the real composition in-vivo could get easily validated.
One can easily imagine the role of threads' specialisation inside the Context()
instance, where transport-class related issues get separated from Context()
's main taks, i.e. doing the house-keeping + managing all the FSA(s) / workflow(s) + holding the asynchronous communication services across its local-interface with the application-side code.
In this very same manner, the transport-class inproc://
, having no real transport-related device to control, can work inside an IO-thread-less Context()
instance, where just memory-mapping works across the "borders" with all due local-process memory, yet without a need to operate any IO-thread.