asynchronousrustlifetime

Does `pub fn long(&self) -> impl Future<Output = ()> + 'static` leak data with every created future?


Basically, building on this answer:

pub fn long() -> impl Future<Output = ()> + 'static {
  let s = String::from("what is the answer?");
  async move {
     // `s` gets moved into this block
     println!("{s}");
  }
}

#[tokio::main]
async fn main() {
    let future = long(); // Create the future
    future.await; // Await the future, it will complete, and `s` will be dropped
}

My question is whether this 'static bound will leak data with every created future? Or does the 'static bound here mean that the trait implementation has to be static? (e.g. where the function definitions are in memory). So far I have:

  1. asked ChatGPT: it says that awaiting drops the future
  2. the paragraph from the book doesn't answer.
  3. read the rustonomicon on lifetime coercion
  4. read similar questions

I'm mainly trying to conceptualize the need and effects of 'static; the code works already.

background

I have a reader that can read some parts, while for another it may need to load additional data into itself before continuing. The best thing I've come up with is:

type Image = Vec<u8>;
#[async_trait]
trait Reader {
    async fn read_into_buffer(&self, offset: u32, buf: &mut [u8]) {
       sleep(2);
       buf.fill(42);
    }
}

struct Decoder<R: Reader> {
  reader: Arc<R>,
  images: HashMap<u32, Arc<Image>>,
}

impl<R> Decoder<R> {
  fn decode_chunk(&self, overview: u32, chunk: usize) -> Result<(),impl Future<Output = Vec<u8>> {
    if let Some(im) = self.images.get(overview) {
      // cheap clones because they are Arcs
      let image = im.clone()
      let r = self.reader.clone()
      async move {
        // don't mention `self` in here
        let buf = vec![0; 42];
        let offset = image[chunk];
        r.read_into_buffer(offset, buf);
        buf
      }
    } else {
        Err(())
    }
  }

  async fn load_overview(&mut self, offset: u32) {
    let buf = vec![0;42];
    self.reader.read_into_buffer(offset, buf).await;
    self.images.insert(offset, buf);
  }
}

which could be used like:

decoder.load_overview(42).await;
let chunk_1 = decoder.decode_chunk(42, 13).unwrap(); // no await'ing
let chunk_2_err = decoder.decode_chunk(13).unwrap_err(); // chunk 13 isn't loaded
decoder.load_overview(13).await;
let chunk_2 = decoder.decode_chunk(13, 13).unwrap();
let res = (chunk_1.await, chunk_2.await)

Solution

  • There is no memory leak. Heap-allocated contents, like s, will be dropped before the .await completes. The stack allocation of the future will go out of scope when .await completes.

    You can imagine the returned Future<Output = ()> + 'static as a state machine, like this enum:

    enum ReturnedFuture {
        Created{s: String},
        Done
    }
    

    This enum, which is 'static on account of no non-'static references, implements Future like this (I've artificially simplified the method signature a lot):

    impl Future for ReturnedFuture {
        fn poll(&mut self) -> Poll<()> {
             match self {
                  Self::Created{s} => {
                      println!("{s}");
                      // Here is where `s` (heap allocation) will be automatically dropped.
                      *self = Self::Done;
                  }
                  Self::Done => {}
             }
             Poll::Ready(())
        }
    }
    

    You can imagine .await being implemented like this (the complexity of yielding Poll::Pending to async runtimes and registering to be woken up via Context's has been elided):

    let future = long();
    // future.await;
    let result = {
        // future is moved to a mutable variable
        let mut fut = future;
        loop {
            let poll = fut.poll();
            if poll.is_ready() {
                break poll;
            } else {
                unreachable!("ReturnedFuture never returns pending, so this complexity has been elided");
            }
        }
        // fut goes out of scope here (stack allocation is dropped).
    };
    // future no longer exists here (it was moved already)
    

    See the comments in the last two code blocks for when freeing of memory happens.