I have to export the CSV data. The volume of data is very high. So I am streaming the response from the microservice. We hit our microservice using dispatcher.
def stream(method: String, urlString: String): Future[Source[ByteString, NotUsed]] =
method match {
case GET =>
val request = Http(url(urlString))
request.map { response =>
response.getStatusCode match {
case StatusOk => Source.single(ByteString(response.getResponseBody))
}
}
}
It will bring all the data. So to fix this issue, I like to modify it and streamed the data from here as well.
I searched a lot and found this question Scala dispatch stream response line by line
But it has no answer.
Thanks and any help will be appreciated.
After searching much, I read it as Input stream and converted to Akka Stream. It worked for me.
def stream(method: String, urlString: String): Future[Source[ByteString, Future[IOResult]]] =
method match {
case GET =>
val futureStream = Http(url(urlString) > as.Response(_.getResponseBodyAsStream))
futureStream.map { inputStream =>
val source = () => inputStream
StreamConverters.fromInputStream(source)
}
}