I have a (somewhat weird) writable stream that I need to convert to a transform stream.
The writable stream, normally, sits at the end of a pipe chain and emits custom events once it has collected enough data for its output. I want it to go in the middle so I can pipe it to another writeStream, i.e:
readStream.pipe(writeStreamToConvert).pipe(finalWriteStream);
What I done is the following and it works.
const through2 = require('through2')
var writeStreamToConvert = new WriteStreamToConvert();
return through2.obj(function (chunk, enc, callback) {
writeStreamToConvert.write(chunk)
// object is the event emitted from the writestream
writeStreamToConvert.on('object', (name, obj ) => {
this.push(JSON.stringify(obj, null, 4) + '\n')
});
callback()
})
This works fine, does not seem to leak memory and is fairly quick. However node gives me a warning:
Warning: Possible EventEmitter memory leak detected. 11 object listeners added. Use emitter.setMaxListeners() to increase limit
So I am a little bit curious if this is the correct way of converting writestreams?
The event handler would be best placed in a Transform stream constructor. Since through2
does not support such initialization, you would need to use node's stream API directly.
Currently, a new event handler (which is never removed -- that is how .on()
works) is being added for every object written to the through2
stream. That is why the warning occurs.