chttpsocketsnetwork-programmingtcp

SHUT_WR and SHUT_RD in the lifecycle of an HTTP connection


I have a TCP socket that performs an HTTP connection with a server. The steps are:

  1. connecting the socket using connect
  2. sending an HTTP request using send
  3. receiving the response using recv

When I shutdown my socket for writing (using shutdown(socket_id, SHUT_WR)) between steps 2 and 3, recv fetches zero bytes. When I try to shutdown my socket for writing after recving, the shutdown function returns Socket is not connected. Solutions that work are:

My questions are:

For completeness, my code where I omitted the shutdown part.

int download_page(url_info* info, http_reply* reply) {
    printf("Downloading page...\n");
    int ret = 0;

    // ===== RESOLVE HOSTNAME INTO IP ADDRESS =====
    struct addrinfo* res;
    struct addrinfo hints;  // specifying we want to establish connexion over TCP
    memset(&hints, 0, sizeof(hints));
    hints.ai_socktype = SOCK_STREAM;
    char* service = (char*) malloc(10);
    sprintf(service, "%d", info->port);
    printf("Resolving host %s on port %s...\n", info->host, service);
    int errcode = getaddrinfo(info->host, service, &hints, &res);
    if (errcode || !res || !(res->ai_addr)) {
        fprintf(stderr, "Could not resolve host %s\n\t%s (%d)", info->host, strerror(errcode), errcode);
        ret = 1;
        goto clean0;
    }
    printf("Host resolved\n");
    
    char* request_message = http_get_request(info);
    printf("Request message:\n%s\nEND of request message\n", request_message);

    // ===== CREATE SOCKET =====
    int socket_id = socket(res->ai_family, SOCK_STREAM, 0);
    if (socket_id == -1) {
        fprintf(stderr, "Could not create socket\n");
        ret = 1;
        goto clean2;
    }
    printf("Socket created\n");

    // ===== CONNECT TO SERVER =====
    if (connect(socket_id, res->ai_addr, res->ai_addrlen)) {
        fprintf(stderr, "Could not connect to server %s\n", strerror(errno));
        ret = 1;
        goto clean2;
    }

    printf("Connected to server :)\n");

    // ===== SEND REQUEST TO SERVER =====
    if (send(socket_id, request_message, strlen(request_message), 0) == -1) {
    //if (write(socket_id, request_message, strlen(request_message)) == -1) {
        fprintf(stderr, "Could not send request to server %s\n", strerror(errno));
        ret = 1;
        goto clean2;
    } else {
        printf("Request sent to server: \"%s\"\n", request_message);
    }

    // ===== RECEIVING RESPONSE =====
    int buffer_size = 1024;
    int remaining_space = buffer_size;
    reply->reply_buffer = (char*) malloc(buffer_size + 1);
    reply->reply_buffer_length = 0;
    if (reply->reply_buffer != NULL) {
        char* buffer = reply->reply_buffer;
        int received;
        do {
            printf("Attempting to receive at most %d bytes from server...\n", remaining_space);
            received = recv(socket_id, buffer, remaining_space, 0);
            if (received == -1) {
                fprintf(stderr, "Could not receive data from server %s\n", strerror(errno));
                ret = 1;
                goto clean2;
            } else if (received != 0) {
                reply->reply_buffer_length += received;
                printf("\tReceived %d bytes, total of %d bytes.\n", received, reply->reply_buffer_length);
                buffer_size *= 2;
                reply->reply_buffer = (char*) realloc(reply->reply_buffer, buffer_size + 1);
                buffer = reply->reply_buffer + reply->reply_buffer_length;
                remaining_space = buffer_size - reply->reply_buffer_length;
            }
        } while (received != 0);
        reply->reply_buffer = (char*) realloc(reply->reply_buffer, reply->reply_buffer_length + 1);
        reply->reply_buffer[reply->reply_buffer_length] = '\0';
        //printf("Received %d bytes from server:\n\"%s\"\n", reply->reply_buffer_length, reply->reply_buffer);
        printf("Received %d bytes from server.\n", reply->reply_buffer_length);
    }

    clean2:
    free(request_message);
    close(socket_id);
    clean0:
    freeaddrinfo(res);
    return ret;
}

The request message is:

GET / HTTP/1.1
Host: www.google.com
Connection: close




Solution

  • what happens when I shutdown the writing end of my socket right after send that causes recv not to get anything?

    Nothing on your end. When you use SHUT_WR, the server receives an EOF when it tries to read again. But an HTTP server shouldn't try to read until after it processes the current request, so you should be able to read the response.

    is shutting down after receiving impossible because HTTP connections are automatically closed after a single request-response chain?

    Connection management is controlled by the Connection: header. In HTTP/1.1, the default is Connection: keep-alive, which means that the connection should be kept open after receiving the response, so it can be reused for the next request.

    the instructions of my assignment say I need to explicitly shut down the writing end of my socket at some point -- why is that necessary and when should I do it?

    If you're not reusing the connection for another request, you should do this after the current request/response is complete.

    If you're sending a request type that includes a body (e.g. POST or PUT), you should be able to shut down the writing end after the body is complete. In fact, you would need to do this if you don't send a Content-Length: header or use "chunked" transfer encoding, as the only way to indicate that the request is complete is by closing the connection. But it shouldn't be necessary for a GET request, since there's no body.