SO_REUSEADDR option not working when application is trying to bind a port

I have an application that reset and start listening again if the socket is closed by remote. The behaviour I am observing is that for the first time, bootup the application is able to listen, but when I try to cleanup and start listening again then I get an error while binding the port. For context, the relevant methods of my server class are below.

void Server::listen() {
    std::cout<<"Server::listen\n";
    m_listenSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
    if (m_listenSocket < 0) {
        std::cout<<"Failed to open listen socket.\n";
        return;
    }
        
    int optval = 0;
    if (setsockopt(m_listenSocket, SOL_SOCKET, SO_REUSEADDR, &optval, sizeof(optval)) < 0) {
        std::cout<<"Failed to set SO_REUSEADDR option.\n";
        close(m_listenSocket);
        return;
    }
        
    struct sockaddr_in server_addr;
    server_addr.sin_family = AF_INET;
    server_addr.sin_addr.s_addr = htonl(INADDR_ANY);
    server_addr.sin_port = htons(m_port);
        
    if (bind(m_listenSocket, (struct sockaddr*)&server_addr, sizeof(server_addr)) < 0) {
        std::cout<<"Failed to bind to port.\n";
        std::cout<<strerror(errno)<<"\n";
        close(m_listenSocket);
        return;
    }
        
    int backlog = 1;  // We are only interested to pool one client.
    if (::listen(m_listenSocket, backlog) < 0) {
        std::cout<<" Failed to listen on the listen socket.\n";
        close(m_listenSocket);
        return;
    }
        
    std::cout<<"Server is listening."<<":port="<<m_port<<"\n";
        
    m_acceptThread = std::thread(&SocketServer::acceptThread, this);
}

void Server::cleanup(){
    std::cout<<"Server::cleanup.\n";
    {
        std::lock_guard<std::mutex> lock(m_mutex);
        m_messageQueue.clear();
        m_sendThreadWait.notify_one();
    }
    ::shutdown(m_listenSocket, SHUT_RD); // socket that is used for listening
    ::shutdown(m_socketId, SHUT_RDWR);// socket that was returned by ::accept
    joinThreads(); // ensures recv/send/accept thread joins and completes
}
    
void Server::resetThread() {
    std::cout<<"Server::resetThread started.\n";
    cleanup();
    {
        std::lock_guard<std::mutex> lock(m_mutex);
        m_startDisconnect = false;
    }
    //sleep(5);<- had no effect
    listen();
    
    std::cout<<"Server::resetThread completed.\n";
}

The output I get is:

Server:listen
Server:listen::SocketServer is listening.:port=11008
...
Server:recvThread::connection closed by remote.
Server:recvThread::completed
Server:resetThread::started.
Server:cleanup
Server:joinThreads
Server:listen
Server:listen:: Failed to bind to port.
Address already in use
Server:resetThread::completed.

Now, once disconnection occurs, my application does reset and will invoke listen() which will try to bind to the same port again, but I have read that SO_REUSEADDR tells the OS that the application can bind immediately again, is that the correct understanding? If so, what might I be missing here?

I also tried putting a 5sec sleep, but that also did not work. Please note that if I wait some time and launch my application again, the application can bind again without any issues.

I am using Linux operating system.

UPDATE:

Showing acceptThread and join thread:

void Server::acceptThread() {
    try {
        std::cout<<"Server::acceptThread::started.\n";
        socklen_t sockaddrLength = sizeof(struct sockaddr_in);
        int incomingSocketId = accept(m_listenSocket, NULL, sockaddrLength);
        if (incomingSocketId < 0) {
            std::cout<<"Failed to accept the client connection\n";
            return;
        }
    
        m_socketId = incomingSocketId;
    
        // Set socket to non-blocking mode
        int flags = fcntl(m_socketId, F_GETFL, 0);
        fcntl(m_socketId, F_SETFL, flags | O_NONBLOCK);
             
        // Start send and recv thread
        m_sendThread = std::thread(&SocketServer::sendThread, this);
        m_recvThread = std::thread(&SocketServer::recvThread, this);
    
    } catch (const std::exception& e) {
        std::cout<<"socket closed by remote."<< e.what()<<"\n";
        return;
    }
    std::cout<<"Server::acceptThread completed.\n";
}

void Server::joinThreads() {
    std::cout<<"Server::joinThreads.\n";
    if (m_acceptThread.joinable()) {
        m_acceptThread.join();
    }
    
    if (m_recvThread.joinable()) {
        m_recvThread.join();
    }
    
    if (m_sendThread.joinable()) {
        m_sendThread.join();
    }
}

Console Output, hangs on accept():

Server:listen
Server:listen::SocketServer is listening.:port=49152
Server:acceptThread::started.
Server:shutdown
Server:joinThreads

I am pretty sure, errno would return EADDRINUSE.

shutdown() does not close the port. You get a leaked socket bound to the port. You should call close() after shutdown() or after joinThreads().