Currently I am building an api gateway in nestjs which consumes a microservice (also in nestjs) responsible to fetch a file from a remote a server as a stream.
Checking the nestjs documentation I found the class StreamableFile, but from what I understood this is to use directly on a gateway.
I have the readable stream on my microservice ready to be returned to the api-gateway, however this should be sent in chunks to the api gateway, and this gateway itself would stream those chunks aswell to the frontend. I don't want to allocate the file in memory neither in microservice or my api-gateway.
My problem is that to communicate with the microservice, I use ClientProxy, and the return from the microservice must be an observable.
Taking this into account I have been trying the following without success:
NestJS Micro-Service:
@MessagePattern('get_file_by_key')
getFileStreamByKey(
keyFileName : string,
): Observable<any> {
const stream = this.remoteServerService.getFile(keyFileName).createReadStream();
return from(stream);
}
NestJS Gateway (Micro-Service consumer)
@Get('/file')
public getFile(): StreamableFile {
return new StreamableFile(this.clientProxy.send('get_file_by_key', 'theFileName'));
}
The above is just to state my thoughts on this, since the code doesn't even compile, because the send method of clientProxy returns an observable and I need a stream for StreamableFile constructor.
Not looking for an immediate solution, but at least some guidance on how this could be implemented and learn from it. Do I need to go with gRPC on my nestjs microservice?
Thank you in advance
The solution I found out was using an extra library to convert a stream to rxjs observable and later on a rxjs observable to stream, using the npm library rxjs-stream.