javascriptnode.jsstreamstreamingnode.js-stream

Why does the 'finish' event fire on a Node.js transform stream before _flush is finished?


I expect the 'finish' the callback to be triggered only once the callback passed to _flush has completed, but it's being triggered before _flush completes. Why?

var stream = require('stream').Transform();

// Nothing interesting here
stream._transform = function (chunk, enc, next) {
  next();
};

// _flush makes an async call.
stream._flush = function (callback) {
  console.log("Expecting 1st: Entering _flush ");

  return process.nextTick(function () {
    console.log("Expecting 2nd: Done with _flush");
    callback();
  });
};

stream.on('finish',function () {
  console.log("Expecting 3rd: Entering finish callback-- should be flushed now.");
});

stream.end("chunk");

The node.js stream documentation for finish says that the event is triggered only when all data has been flushed to the underlying system.

Looking at the source code for Node's Transform stream implementation it looks like the intent is for finish to be triggered only when the _flush callback is called:

this.once('prefinish', function() {
  if (util.isFunction(this._flush))
    this._flush(function(er) {
      done(stream, er);
    });
else
   done(stream);
});

Either I'm missed something or this is a bug with Node's Transform streams. I assume it's the former but haven't spotted my issue. Thanks for your help!

Update: Note that if I replace the call to process.nextTick() with a synchronous call to sleep, as the sleep module provides, then the issue goes away. The issue is only triggered by async calls within _flush.


Solution

  • In node v0.10, _flush() is called on 'finish', so that is more obvious to see that that is what is expected.

    In master (v0.11.x) however, it looks like a new 'prefinish' event was invented and changes things a little bit. However if you look at the code, you'll see that prefinish() is called which triggers _flush(). When prefinish() returns after emitting 'prefinish', 'finish' is then emitted. It does not wait for the _flush() callback to be called before doing this.

    So the resulting behavior is the same for both versions. Now whether the behavior should be changed is a separate issue. You might create a github issue about the situation and see if there is any interest in getting the behavior modified.