javascriptsendgridmultipartmultiparty

Multipart form, order of fields/files


My API needs to handle file uploads as well as attached files in emails. Emails are routed to my API via the mail parse function at Sendgrid.

Both methods results in a multipart form. In the email case, fields are "to", "from", etc. In the ordinary upload form I decide the fields.

I don't want to handle local temp files, so ideally I'll just pipe the file streams to S3. I have implemented this and it works great. But there are edge cases..

Before I pipe the files to S3 I want to ensure that the email is legit. I do this by checking both the from and to addresses. Unfortunately multipart forms does not specify a specific order for fields/files, rather the spec says to preserve order of original form.

Problem

Occationally I have received emails from Sendgrid parse where the fields arrive AFTER the files.

So, in order to validate the sender via the form fields I might need to buffer the file while waiting for the field data. This seems to be a quite stupid way of handling incoming files, i.e. start to process/store files before I even know if I should handle them.

All emails sent to a specified subdomain will be forwarded from Sendgrid to my server. This means that I could potentially be flooded with emails which I'd have to handle completely (including buffering the files) before I can decide if the email is legit.

Question

Have I misunderstood something?

Some email clients postpones downloading of attachments. How is this done?

The tools I'm using


Solution

  • At last I grasped this. The simple answer is that a multipart form is a single stream. If files comes before fields in the stream I have to buffer the files while awaiting the fields. At least if I need the information from fields in order to process the files.

    I guess that email clients which are able to postpone downloading of attachments must rely on a server that parses the email and buffers the files.

    Thank you Doug Wilson at https://github.com/pillarjs/multiparty for clarifying this.