We have an application that uploads a PDF to S3 and returns an object with metadata about the S3 file. This works fine if we want to email the link to the document to someone. The problem comes if we want to download the file from S3 immediately thereafter, for instance, to attach the file to a message vs. a link to the file. What seems to be happening is that the metadata regarding the S3 file is returned to the calling function BEFORE the pdf.on('end') code is executed which triggers the upload to S3. The relevent code snippet is here:
pdf.on('end', async () => {
const result = await Buffer.concat(chunks);
await s3.putObject(
{
Bucket: 'corp-attch',
Key: filePath,
Body: result,
ContentType: 'application/pdf',
}).promise();
this.logger.log(`S3 Upload Success: ${filePath}`);
});
await pdf.end();
this.logger.log(`Finished with pdf: ${filePath}`);
returnAttach.attachDest = filePath;
returnAttach.attachName = filePath.split('/').pop();
const getObjectParams = {
Bucket: this.configService.awsCorpBucketName,
Key: filePath, Expires: 900,
};
returnAttach.attachUrl = s3.getSignedUrl('getObject', getObjectParams);
return new Promise(resolve => resolve(returnAttach));
Is there a good way to force the whole thing to wait until the s3.putObject is complete?
In this case, no matter what we try in this code, the 'S3 Upload Success' message is logged AFTER the 'Finished with pdf:' code and any attempt to access the file via the url returns a 404 when we try to do so from the calling function.
I'm assuming pdf.end()
is what eventually triggers the pdf.on('end',
event - I can't find any documentation in pdfkit
regarding this end
event, so, based on the fact that it must be triggered for your code to at least work as it does, I'm assuming it is valid.
Note: I've removed the superfluous await
for Buffer.concat
and the bizarre return new Promise
at the end - you just need to return a value inside an async
function, since an async
function always returns a Promise
anyway
This could do what you require.
await new Promise(resolve => {
pdf.on('end', resolve);
pdf.end();
});
const result = Buffer.concat(chunks);
await s3.putObject({
Bucket: 'corp-attch',
Key: filePath,
Body: result,
ContentType: 'application/pdf',
}).promise();
this.logger.log(`S3 Upload Success: ${filePath}`);
this.logger.log(`Finished with pdf: ${filePath}`);
returnAttach.attachDest = filePath;
returnAttach.attachName = filePath.split('/').pop();
const getObjectParams = {
Bucket: this.configService.awsCorpBucketName,
Key: filePath,
Expires: 900,
};
returnAttach.attachUrl = s3.getSignedUrl('getObject', getObjectParams);
return returnAttach;
To explain the first 4 lines of code
This creates (and awaits) a new Promise
that resolves once the end
event is fired. pdf.end()
asynchronously triggers that event.
I'm not sure, but I think it could also be written
pdf.end();
await new Promise(resolve => {
pdf.on('end', resolve);
});
Since the event is triggered asynchronously
I don't know pdfkit very well, been years since I even looked at it, I wonder if there's a error
event also, since the code as presented won't handle errors at all (will just wait eternally for that Promise to resolve)
I wonder if
await new Promise((resolve, reject) => {
pdf.on('end', resolve);
pdf.on('error', reject);
pdf.end();
});
Would be more complete for error handling.
I also just realised that this is node.js
- so you could probably do
import { once } from 'node:events';
pdf.end();
await once(pdf, "end");
.... etc