node.jsamazon-web-servicesamazon-s3fs

Read file from aws s3 bucket using node fs


I am attempting to read a file that is in a aws s3 bucket using

fs.readFile(file, function (err, contents) {
  var myLines = contents.Body.toString().split('\n')
})

I've been able to download and upload a file using the node aws-sdk, but I am at a loss as to how to simply read it and parse the contents.

Here is an example of how I am reading the file from s3:

var s3 = new AWS.S3();
var params = {Bucket: 'myBucket', Key: 'myKey.csv'}
var s3file = s3.getObject(params)

Solution

  • You have a couple options. You can include a callback as a second argument, which will be invoked with any error message and the object. This example is straight from the AWS documentation:

    s3.getObject(params, function(err, data) {
      if (err) console.log(err, err.stack); // an error occurred
      else     console.log(data);           // successful response
    });
    

    Alternatively, you can convert the output to a stream. There's also an example in the AWS documentation:

    var s3 = new AWS.S3({apiVersion: '2006-03-01'});
    var params = {Bucket: 'myBucket', Key: 'myImageFile.jpg'};
    var file = require('fs').createWriteStream('/path/to/file.jpg');
    s3.getObject(params).createReadStream().pipe(file);