I want to write a multithreaded TCP listener using epoll and EPOLLET
.
I have seen that there are several possibilities:
Each thread has its own epoll fd, does bind()
using SO_REUSEPORT
(but only up Linux kernel 3.9) and handles its own connections. EPOLLONESHOT
would not be needed in this case because each thread handles its own file descriptors.
There is a main thread which accepts connections and several worker threads which handle those connections. Each worker thread has its own epoll fd. How can the main thread in this case fairly distribute the connections among the worker threads? It could add the file descriptor to the epoll fd of another thread using a round-robin fashion (but it might happen that that particular thread is still busy handling another connection). Or connections could be added to a global queue, and the main thread would use pthread_cond_signal()
, but then we need a mutex and a condition variable.
There is a main thread which accepts connections and several worker threads which handle those connections. There is a global epoll fd, EPOLLONESHOT
would then be needed in this case, so not all the threads get woken up for the same event.
I know that, if EPOLLET
is used, once I get notified about an event, I have to drain the fd, until I get EAGAIN
.
If the socket option SO_REUSEPORT
is not supported (older kernel), which option would be the best?
SO_REUSEPORT
would be to have a common epoll fd and a common listener which are shared among all the threads. EPOLLONESHOT
is required so only one thread handles the events for a certain fd at a time.