udp select timeout issues. Either timing out or reading from all clients

I am using select to handle connections on a udp server. If I do not get a packet for some period, I would like to time out. The probelm is, it seems I can either timeout correctly and only read from one client, or read from all clients and not time out.

The difference in this functionality has to do with the first argument to select, the int nfds

Here is my code:

int TIMEOUT = 5;
for (;;) {
    FD_ZERO(&read_handles); 
    FD_SET(udpFD, &read_handles); 

    timeout.tv_sec = TIMEOUT;
    timeout.tv_usec = 0;
    if (select(udpFD+1, &read_handles, NULL, NULL, &timeout) == 0) {
        printf("Select has timed out...\n");
        return 1;
    } else {
        int length = 1;
        if (FD_ISSET(udpFD, &read_handles)) {
           //process read.
        }
    }
}

This version does not time out. If I change the select line to:

if(select(udpFD, &read_handles, NULL, NULL, &timeout) == 0)

It does timeout, but it only receives data from one of my clients.

udpFD is the only handle I am looking at, but it has a value of 4 because it is not the first descriptor I have made. I do not know if that makes a difference because it is the max value.

How can I both timeout and get data from both of my clients?

Using if(select(udpFD+1, &read_handles, NULL, NULL, &timeout) == 0) is the correct way to go.

This will work.

My error was later in the code I was not resetting a length field I read, and was getting stuck in the recvfrom loop, and only calling select once.