When should I close the socket, if the request contains Connection: keep-alive?

When I am using this Python script to send a request to my server:

import requests as r

url = "http://localhost:8070/"
response = r.get(url=url)

It sends the following request:

GET / HTTP/1.1
Host: localhost:8070
User-Agent: python-requests/2.27.1
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive

If I understand it correctly, Connection: keep-alive means that I should not close the client-socket, because the client could use it again.

But if I do not close the client-socket, the Python script just gets stuck until I close the socket. Is there another way to indicate that the request is complete so pythons requests understands it?

If I try to send the request to any other server, the script almost immediately finishes. My guess would be to timeout the client after a few milliseconds e.g. by using select like this:

struct timeval timeout;

timeout.tv_sec = 0;
timeout.tv_usec = 1000;
select_ret = select(this->_maxfds + 1, &this->_readfds, &this->_writefds, NULL, &timeout);

And now I would just close the client-socket after select returns 0:

if (select_ret == 0) {
   close(client_socket);
}

Is that a valid approach, or am I missing something?

I am sending the response like this:

char *response = "HTTP/1.1 200 Ok\r\n\r\n";
send(this->_client_socket, response, strlen(response), 0)

BUT this does not terminate the python script. The python script still hangs after I execute this line of code. It only finishes when I close the socket on my side.

So how would I determine if I should close it or not? As I already said my approach was to use a timeout in case no data is getting written in to the socket from the client side.

Solution

Your server's response is incomplete.

Your understanding of Connection: keep-alive in the request is correct. However, there is no Content-Length or Transfer-Encoding: chunked header present in your response, so the only way the client has to know when the response is finished is to wait for the socket connection to be closed on the server side. Read the rules outlined in RFC 2616 Section 4.4 and RFC 7230 Section 3.3.3 of the HTTP 1.1 protocol spec.

Try something more like this instead:

const char *response = "HTTP/1.1 200 Ok\r\nContent-Length: 0\r\n\r\n";
send(this->_client_socket, response, strlen(response), 0)