node.jsexpressamazon-s3multermulter-s3

Upload to S3 using multer without a middleware function


I am using multer to upload media to my s3 bucket. I am using multer-s3 as a middleware to upload media like:

const upload = multer({
  storage: multerS3({
    s3: s3,
    bucket: myBucket,
    key: function (req, file, cb) {
      cb(null, new Date().getTime() + '_' + file.originalname)
    }
  })
});

and calling it in route as:

router.post("/media",upload.single("media"))

This works great. But a scenario that is not working for me is: Let say i upload one image and i want to store multiple version of it by resizing it before uploading. I am not able to call the upload function like a normal one. I want to do something like:

let thumbnail = myFunctionToReturnImageFile(req.file);
upload(thumbnail);

I understand i need to send some multi-part/form-part but i am unable to find a solution. If you suggest me a something that will be great.


Solution

  • Option 1 - Use multer-s3-transform middleware to handle S3 upload with file transformations, like image resizing using sharp.

    const multer = require('multer')
    const multerS3 = require('multer-s3-transform')
    const sharp = require('sharp')
    const AWS = require('aws-sdk')
    const S3 = new AWS.S3({
        accessKeyId: ...,
        secretAccessKey: ...
    })
    
    const upload = multer({
      storage: multerS3({
        s3: S3,
        bucket: ...,
        shouldTransform: true,
        transforms: [
          {
            id: 'original',
            key: (req, file, cb) => cb(null, new Date().getTime() + '_' + req.file.originalname),
            transform: (req, file, cb) => cb(null, sharp().jpg())
          },
          {
            id: 'large',
            key: (req, file, cb) => cb(null, new Date().getTime() + '_large_' + req.file.originalname),
            transform: (req, file, cb) => cb(null, sharp().resize(1200, 900).jpg())
          },
          {
            id: 'small',
            key: (req, file, cb) => cb(null, new Date().getTime() + '_small_' + req.file.originalname),
            transform: (req, file, cb) => cb(null, sharp().resize(400, 300).jpg())
          }
        ]
      })
    })
    
    router.post('/media', upload.single('media'))
    

    Option 2 - If you insist on doing things manually, you may use plain multer to handle the incoming file, resize it with sharp and upload each resized file to S3 using AWS-SDK.

    const multer = require('multer')
    const sharp = require('sharp')
    const AWS = require('aws-sdk')
    const S3 = new AWS.S3({
        accessKeyId: ...,
        secretAccessKey: ...
    })
    
    router.post('/media', multer().single('media'), (req, res) => {
      // req.file represents the uploaded file
      let time = new Date().getTime()
      
      Promise.all([
        // original file
        S3.upload({
          Bucket: ...,
          Key: time + '_' + req.file.originalname,
          Body: req.file.buffer
        }),
        
        // large preview
        sharp(req.file)
          .resize(1200, 900)
          .toBuffer()
          .then(resized => S3.upload({
            Bucket: ...,
            Key: time + '_large_' + req.file.originalname,
            Body: resized
          }).promise()),
        
        // small thumbnail
        sharp(req.file)
          .resize(400, 300)
          .toBuffer()
          .then(resized => S3.upload({
            Bucket: ...,
            Key: time + '_small_' + req.file.originalname,
            Body: resized
          }).promise()),
      ])
      .then(() => res.send("Images uploaded"))
      .catch(e => {
        console.warn(e) // debug this error
        res.status(500).send("Unable to upload images")
      })
    })