I am writing a static file server with hyper 1.0 and tokio 1.0. When serving a file, I want it to be as fast as possible while allocating the minimum required amount of memory. In other words: I want to stream the file directly from disk to the network, without reading it first.
Specifically: I have a tokio::fs::File and want to turn it into a hyper Body.
use std::path::Path;
async fn handle(path: &Path) -> hyper::Response<???> {
let file = tokio::fs::File::open(path).await.unwrap();
// TODO: what here?!
Response::builder().body(body).unwrap()
}
I know of hyper-body-util, but all the Body, Stream, AsyncRead, Frame terms confuse me.
There are a few keys to this:
tokio::fs::File implements AsyncRead, which is similar to std::io::Read in that it can be instructed to read a bunch of data into a buffer.tokio_util::io::ReaderStream "converts" an AsyncRead to a Stream of Bytes. These two traits have slightly different semantics and hyper can only deal with Streams.Frames, not just Bytes. Luckily, it's easy to convert via Frame::data(bytes). Using TryStreamExt::map_ok, we can turn our stream into a stream of frames.http_body_util::StreamBody to finally get something that implements hyper::Body.map_ok. To make working with it easier, we convert it into a http_body_util::combinators::BoxBody. The dynamic dispatch overhead is insignificant compared to the actual IO operations.All that put together looks like this:
use bytes::Bytes;
use futures::TryStreamExt;
use http_body_util::{combinators::BoxBody, StreamBody};
use hyper::body::Frame;
use std::path::Path;
use tokio_util::io::ReaderStream;
type FileStreamBody = BoxBody<Bytes, std::io::Error>;
async fn handle(path: &Path) -> hyper::Response<FileStreamBody> {
let file = tokio::fs::File::open(path).await.unwrap();
let byte_stream = ReaderStream::new(file);
let frame_stream = byte_stream.map_ok(Frame::data);
let body = StreamBody::new(frame_stream);
let boxed_body = BoxBody::new(body);
hyper::Response::builder().body(body).unwrap()
}