I am using AWS Lambda service to read the content from the CSV file uploaded on the S3 bucket and convert it to the JSON format. I got stuckup on the CSV to JSON phase.
const AWS = require('aws-sdk');
const csvtojson = require('csvtojson');
const S3 = new AWS.S3();
exports.handler = async (event,content,callback) => {
console.log(' inside trigger ')
const src_bkt = event.Records[0].s3.bucket.name;
const src_key = event.Records[0].s3.object.key;
try {
const params = {Bucket: src_bkt, Key:src_key}
// get csv file and create stream
const stream = S3.getObject(params).createReadStream();
// convert csv file (stream) to JSON format data
const json = await csvtojson().fromStream(stream);
console.log(json);
}
catch (err) {
console.log(JSON.stringify(err))
return {
statusCode: err.statusCode || 400,
body: err.message || JSON.stringify(err.message)
}
}
};
Error Message: "errorType": "Runtime.ImportModuleError", "errorMessage": "Error: Cannot find module 'csvtojson'",
I tried to convert the node module on the lambda to check any luck.
You need to package and deploy the lambda as a zip file. The file should contain the javascript file with the handler function as well as the node_modules
directory with all the dependencies.
handler.js
node_modules/
......................../csvtojson
my-project
using npm init
, it will create a package.json file for youhandler.js
install csv2json using npm install csv2json --save
, now you will see a node_modules
directory created
zip the my-project
directory, so you have my-project.zip, the zip should directly contain the handler.js
and node_modules
Reference:
https://aws.amazon.com/premiumsupport/knowledge-center/lambda-deployment-package-nodejs/
hope this helps.