node.jspipechild-processstdio

Node Child Process understanding additional STDIO pipes beyond 0, 1 and 2 in SPAWN method


I am currently upping my understanding on child processes (only spawn-method) in NodeJS by pondering over the respective manual and many other sources.

All sources (which I have seen so far) only discuss the default three pipes to the parent process; stdin, stdout, and stderr.

These correspond to, or can be configured or addressed in Options thus:

Example ['ignore', 'inherit', 'pipe'] or [0, 1, 2] etc.

I have however come across code a couple of times, where additional pipes are being used. Let's say in the SPAWN method:

Spawned.stdio[3].on('data', (data) => {
   PRESTREAM.write(data, 'binary');
   });

with Options like:

const Options = {
   detached: true,
   stdio: ['ignore', 'ignore', 'ignore', 'pipe']
   };

Stdio[3] and the 4rth stdio option is what is confusing me here.

Question: I am not trying achieve any specific coding goal or result. I am looking for an explanation or a pointer to helpful documentation to get my head around this part, prior to using child processes in an eminent development project with CPU intensive tasks.

When is it a good idea to configure additional pipes beyond the standard ones? How is this to be implemented?

Any hints would be much appreciated tyvm!

Update

Of course I am assuming that(!) these options configure an additional stdout that is being piped to the writing PRESTREAM function and have tested that succesfully.

Without any documentation of this I am a bit vary about running into problems deep into the development road. Stdio functionalies of 0-2 are all clearly defined (stdin, stdout and stderr).

Are further stdios completely freely configurable as additional stdouts or do they have special or reserved purposes?


Solution

  • Since there isn't an answer to this in over a months time, I am assuming the subject may be of the rarer sort.

    In that case it may be beneficial if I added my findings after having tested for over 3 weeks - despite not being an authority on the subject.

    Please feel free to correct me on anything and I will update this answer.

    Yes - it is possible to define additional STDIO's beyond the standard three ones.

    This is achieved by adding a fourth (or more) entry to the STDIO array in the child process options like so:

    const Options = {
       detached: true,
       stdio: ['ignore', 'ignore', 'ignore', 'pipe']
       };
    

    Benefits

    When using the detached option to start a long-running process, the process will not stay running in the background after the parent exits unless it is provided with a stdio configuration that is not connected to the parent. If the parent process' stdio is inherited, the child process will remain attached to the controlling terminal.

    https://nodejs.org/api/child_process.html

    The official resource above offers some further information on beneficial behavior of truly detaching and / or unreffing ( subprocess.unref(); ).

    The attachment described above, extends into an easily overlooked, but for my application icomparably more important performance issue;

    In fact I have found, that in some cases, the expected, desired, beneficial effect of spinning up seperate processes may not even be achievable, without completely shutting down any connections between the parent process and the three standard STDIOs.

    I am handing over (piping) the respective stdout to another child process.

    In my tests I experienced drops in performance during any attempt of using a pipe from the standard stdout, when compared to simply having two additional detached, spawned child processes, with all ties to the parent process cut (ignore), piping one to another and / or other processes that are independant of the parent process (OS processes, File System etc).

    Hopefully this helps future readers, until someone more knowledgable has some further input.