pipeforkstdinexecvpdup2

Executing sed via execvp makes other pipes blocked


I am playing around with fork/pipe/dup/execvp concepts and came out with a little problem:

I made a test main to execute sort and manually send some numbers via pipe bound to STDIN and it works as expected: When I close the stdin writer, I can read the sorted numbers output.

However, if I also start sed (or other program which listens on STDIN) in the same way with his own pair of pipes (not connected to the ones used in sort), then I got a block on read from sort's pipe that is gone if I just comment the sed execution part.

The idea is to be able to read from "first" one and have the others waiting for command sending.

So the questions are: a) Do they share the same STDIN so then when sed starts it counts as a possible writter and this is why read blocks? and b) How then I can spawn those N process from the same parent that make their STDIN undependant from each other?

This is the test code:

#include <iostream>
#include <unistd.h>
#include <wait.h>

int sed_pipe_in[2], sed_pipe_out[2], sort_pipe_in[2], sort_pipe_out[2];

pid_t execute_sort() {
    pid_t sort = fork();

    if (sort == 0) {
        close(sort_pipe_in[1]);
        dup2(sort_pipe_in[0], STDIN_FILENO);
        close(sort_pipe_in[0]);

        close(sort_pipe_out[0]);
        dup2(sort_pipe_out[1], STDOUT_FILENO);
        close(sort_pipe_out[1]);

        char* args[] = { "sort", "-", NULL };
        execvp("/usr/bin/sort", args);
        std::cerr << "Execvp failed!" << std::endl;
        exit(-1);
    }
    else if (sort == -1) {
        std::cerr << "Fork failed!" << std::endl;
        exit(-1);
    }
    else {
        close(sort_pipe_in[0]);
        close(sort_pipe_out[1]);
    }
    return sort;
}

pid_t execute_sed() {
    pid_t sed = fork();

    if (sed == 0) {
        close(sed_pipe_in[1]);
        dup2(sed_pipe_in[0], STDIN_FILENO);
        close(sed_pipe_in[0]);

        close(sed_pipe_out[0]);
        dup2(sed_pipe_out[1], STDOUT_FILENO);
        close(sed_pipe_out[1]);

        char* args[] = { "sed", "-e", "s/3/9/", NULL };
        execvp("/bin/sed", args);
        std::cerr << "Execvp failed!" << std::endl;
        exit(-1);
    }
    else if (sed == -1) {
        std::cerr << "Fork failed!" << std::endl;
        exit(-1);
    }
    else {
        close(sed_pipe_in[0]);
        close(sed_pipe_out[1]);
    }
    return sed;
}

int main() {
    pipe(sed_pipe_in);
    pipe(sed_pipe_out);
    pipe(sort_pipe_in);
    pipe(sort_pipe_out);

    pid_t sort = execute_sort();
    // Why removing the following line makes the program to work
    // if sed is not yet connected to sort?
    pid_t sed = execute_sed();
    
    
    // Only parent will get here
    // Sending input to sort
    write(sort_pipe_in[1], "3\n2\n1\n", 7);
    close(sort_pipe_in[1]);

    char buffer[500];
    // If sed is started then this read is blocking
    ssize_t bytes = read(sort_pipe_out[0], buffer, 500);
    std::string output(buffer, bytes);

    std::cout << "Output is " << output << std::endl;

    // Sort will exit here as stdin is closed
    int status;
    waitpid(sort, &status, 0);
    return 0;
}

If execute_sed is commented out, then the output is correct:

1
2
3

but if execute_sed is not commented then the process is blocked at read.


Solution

  • For each of your sort and sed forks, the other fork is keeping file descriptors for the other's process's pipes open (as forked children inherit file descriptors).

    Since sort's STDIN is still open (and held by the sed process), it continues waiting for input.

    You can either just have each one close() the unneeded file descriptors, create the pipes using pipe2(res_pipe, O_CLOEXEC) (which will make them close automatically on exec*), or use fcntl with F_GETFD/F_SETFD to mark the pipe file descriptors with FD_CLOEXEC for the same result.