node.jsexpressfsmulter

Consume content of the file without saving the file with express.js?


I am writing an application where I have an CVS file upload. I have an CVS parser so from my frontend I can upload the CSV to backend, process it and save it to database. After that I am deleting the file.

I am using multer to accept the file and it get's saved to hard drive, then I can read the file and consume the content of the file and delete the file. All fine and all good there.

I am trying to figure out if there is a way to skip actually saving the file altogether. Is there an easy way of just submitting the file from 'multipart/form-data' form and reading the content directly with express without having to save it on file system?

Just for reference, this is what I have now and it is working as expected

On frontend:

static fileUpload(file) {
  const url = '/api/statement-upload';
  const formData = new FormData();
  formData.append('file', file);
  const config = {
    headers: {
      'content-type': 'multipart/form-data'
    }
  };
  return post(url, formData, config);
}

On my express server I'm using multer and have defined route like this:

import multer from 'multer';

export default function (router) {
  const upload = multer({dest: 'uploads/'});

  router.use('/api/statement-upload', upload.single('file'), (req, res) => {
    cvsParser(req.file.path)
      .then(() => res.json({error: false}))
      .catch(e => res.json({error: e}));
  });

  return router;
};

And in my cvsParser then I have this:

export default function (filePath) {
  let content = fs.readFileSync(filePath, 'binary');  
  fs.unlink(filePath, () => {});

  // do nasty stuff to content and save it gently in the database
}

So in short again, is there a way to do this without having to resort to saving the CSV to file system before I can consume the content of the file?

Is this even possible considering the file encodings, etc, maybe there is no way around using fs?


Solution

  • This is what can be done when you don't want to store the csv in a file system and read the content. In my case, I had to pass the content of csv file as a string to another server (without saving in local system of server).

    const multer = require('multer');
    const upload = multer({ storage: multer.memoryStorage() })
    
    app.post(
            '/users/bulkUpload',
            upload.single('csvFile'),
            this.usersRoutes.uploadUserData
     );
    

    and in uploadUserData():

    uploadUserData = async(
        req: Request,
        res: Response,
        next: any
    ): Promise<any> => {
        try {
            const options = {
                    formData : {
                        'upload': String(req.file.buffer)
                    },
                    headers: {
                        authorization: req.token,
                        'Content-type': 'multipart/form-data'
                    },
                    json: true
            };
            const response = await this.http.post.exec(
                    '/somePostUrl/',
                    options
            );
            return res.status(200).json(response.body);
        }catch (error) {
            return next(error);
        }
    }
    

    Here, console.log(String(req.file.buffer)) would show you the content of csv file as a string.

    I hope it helps you.