javascriptnode.jsasync-awaitecmascript-2017

How to use ES8 async/await with streams?


In https://stackoverflow.com/a/18658613/779159 is an example of how to calculate the md5 of a file using the built-in crypto library and streams.

var fs = require('fs');
var crypto = require('crypto');

// the file you want to get the hash    
var fd = fs.createReadStream('/some/file/name.txt');
var hash = crypto.createHash('sha1');
hash.setEncoding('hex');

fd.on('end', function() {
    hash.end();
    console.log(hash.read()); // the desired sha1sum
});

// read all file and pipe it (write it) to the hash object
fd.pipe(hash);

But is it possible to convert this to using ES8 async/await instead of using the callback as seen above, but while still keeping the efficiency of using streams?


Solution

  • The await keyword only works on promises, not on streams. There are ideas to make an extra stream-like data type that would get its own syntax, but those are highly experimental if at all and I won't go into details.

    Anyway, your callback is only waiting for the end of the stream, which is a perfect fit for a promise. You'd just have to wrap the stream:

    var fd = fs.createReadStream('/some/file/name.txt');
    var hash = crypto.createHash('sha1');
    hash.setEncoding('hex');
    // read all file and pipe it (write it) to the hash object
    fd.pipe(hash);
    
    var end = new Promise(function(resolve, reject) {
        hash.on('end', () => resolve(hash.read()));
        fd.on('error', reject); // or something like that. might need to close `hash`
    });
    

    There also exists a helper function to do just that in more recent versions of nodejs - pipeline from the stream/promises module:

    import { pipeline } from 'node:stream/promises';
    const fd = fs.createReadStream('/some/file/name.txt');
    const hash = crypto.createHash('sha1');
    hash.setEncoding('hex');
    
    // read all file and pipe it (write it) to the hash object
    const end = pipeline(fd, hash);
    

    Now you can await that promise:

    (async function() {
        let sha1sum = await end;
        console.log(sha1sum);
    }());