rust

Extract value from nested Box<T>


In my project I have recursive enum that holds various constraints. One of the variants implies that constraint is negated.

#[derive(PartialEq, Eq, PartialOrd, Ord, Debug)]
enum A {
    NOT(Box<A>),
    B
}

I need to implement optimization function that will get rid of double negation:

let mut x = A::NOT(Box::new(
    A::NOT(Box::new(
      A::B
    ))
));
x.optimize(); // x == A::B

I'm struggling with proper extraction of Boxed values. My implementation so far:

impl A {
    fn optimize (&mut self) {
        let replacement = match self {
            A::NOT(a1) => {
                match &**a1 {
                    A::NOT(a2) => Some(a2),
                    _ => None
                }
            },
            _ => None
        };
        
        if let Some(a) = replacement {
            *self = **a;
        }
    }
}

Throws error[E0507]: cannot move out of **a which is behind a shared reference. My feeling is that I have to destroy Boxes first. After a brief research I've stumbled upon into_raw. But I have no idea how to handle raw pointers it returns and how to address "the caller should properly destroy T" requirement of this function.


Solution

  • Now the question is reopen, here is the solution suggested in an earlier comment.
    It uses std::mem::replace() only if two A::NOT are first seen in a row (no need to undo the replacement, which I find easier to read).

        fn optimize(&mut self) {
            if let A::NOT(a1) = self
                && let A::NOT(a2) = a1.as_mut()
            {
                *self = std::mem::replace(a2, A::B);
            }
        }
    

    The idea is to steal the content of the second A::NOT, since this is what we want to keep as self.
    Unfortunately, this would leave this second A::NOT in an inconsistent state (it cannot be empty) even if it would be dropped right after.
    Usually, std::mem::take() serves this purpose, storing the default value of the stolen type instead, but in this specific case, the type A does not provide a default value; instead, we use std::mem::replace() in order to provide a dummy value, cheap to build and destroy immediately.
    Actually, as soon as *self is reassigned with the stolen value, its former content is dropped: the first A::NOT then the second one, now containing the dummy value.

    Note that this formulation makes use of if-let-chains, which were introduced in Rust 1.88.
    With an earlier version of the compiler, the && let should be replaced with a nested if-let.