I'm trying to adapt the data model I designed to a serde MapAccess interface. I have my own data model, which has a List type containing key-value pairs, which are exposed like this:
impl List {
fn next_item(&mut self) -> Option<(String, Item<'_>)>
}
In other words, the Item type mutably borrows from a List object, and must be fully processed before the next Item can be extracted from the List.
I need to adapt this data model to a deserializer trait that looks like this:
trait MapAccess {
fn next_key<T: Deserialize>(&mut self) -> Option<T>;
fn next_value<T: Deserialize>(&mut self) -> T;
}
The two methods should be called in alternating order; they're allowed to panic or return garbage results if you don't call them in the correct order. This trait is provided by the deserializer library I'm using, and is the one only of this problem I can't control or modify.
I'm trying to adapt a List or an &mut List into a MapAccess and I'm running into challenges. I do have a function which converts an Item<'_> to a T: Deserialize (and specifically ends the borrow of List):
fn deserialize_item<T: Deserialize>(item: Item<'_>) -> T;
The basic problem I'm running into is that MapAccess::next_key calls List::next_item and then deserializes the String as a key, and then needs to store the Item<'_> so that it can handled by next_value. I keep running into cases with types that are almost self referential (two fields that both mutably borrow the same thing). It's nothing truely self-referential so I'm not (yet) convinced that this is impossible. The closest I've come is some kind of enum that that resembles enum State {List(&'a mut List), Item(Item<'_>)} (because I don't need to access a List while I'm working on the Item, but I haven't found a way to restore access to the List for the next next_key after the Item is consumed.
As the author of the data model, I do have some leeway to change how it's designed, but if possible I'd like to hold on to some kind of borrowing model, which I use to enforce parse consistency (that is, the borrow prevents List from parsing the next item until after the Item has fully parsed the content of the current item.). I can of course design any additional types to implement this correctly.
Note: This is all serde related, but I'm using original types to simplify the interfaces. In particular I've omitted all the of the Result types and 'de lifetimes as I'm confident they're not relevant to this problem.
EDIT: Here's my attempt to make something that works. This doesn't compile, but it hopefully gets across what I'd like to do here.
fn deserialize_string<T: Deserialize>(value: String) -> T { todo!() }
fn deserialize_item<T: Deserialize>(item: Item<'_>) -> T { todo!() }
struct ListMapAccess<'a> {
list: &'a mut List,
item: Option<Item<'a>>,
}
impl<'a> MapAccess for ListMapAccess<'a> {
fn next_key<T: Deserialize>(&mut self) -> Option<T> {
let (key, item) = self.list.next_item()?;
self.item = Some(item);
Some(deserialize_string(key))
}
fn next_value<T: Deserialize>(&mut self) -> T {
let item = self.item.expect("Called next_item out of order");
deserialize_item(item)
}
}
Now, I'm really not at all tied to a ListMapAccess struct; I'm happy with any solution that integrates my List / Item interface into MapAccess.
This appears to be impossible in safe rust, but I've come up with this unsafe solution.
// `List` is a type that I control that I'd like to adapt to
// `MapAccess`.
impl List {
fn next_item(&mut self) -> Option<(String, Item<'_>)>
}
// Fake version of `serde::de::MapAccess`. I don't have control
// of this trait.
trait MapAccess {
fn next_key<T: From<String>>(&mut self) -> Option<T>;
fn next_value<T: From<Item>>(&mut self) -> T;
}
struct ListMapAccess<'a> {
fake_list: PhantomData<&'a mut List>,
list: NonNull<List>,
item: Option<Item<'a>>,
}
/// Returned if you call `next_item` while `item` is not `None`
/// or if you called `use_item` while it *is* `None`
struct BorrowError;
impl<'a> ListMapAccess<'a> {
fn next_item(&mut self) -> Result<Option<String>, BorrowError> {
if self.item.is_some() { return Err(BorrowError) }
// SAFETY: It's only safe to create this reference if the `item` is
// empty, because the `item` borrows from the list. If `item` is
// empty, then there are no "real" borrows of `List` in existence,
// since we're pretending to own a mutable borrow in `ListMapAccess`.
let list: &'a mut List = unsafe { self.list.as_mut() };
let Some((key, item)) = list.next_item() else { return Ok(None) };
self.item = Some(item);
Ok(Some(key))
}
fn use_item<R>(
&mut self,
// The use of `for<'b>` here ensures that `op` can't
// cause the `Item` to outlive `ListMapAccess`.
op: impl for<'b> FnOnce(Item<'b>) -> R
) -> Result<R, BorrowError> {
self.item.take().map(op).ok_or(BorrowError)
}
}
impl MapAccess for ListMapAccess<'_> {
fn next_key<T: Deserialize>(&mut self) -> Option<T> {
self.next_item()
.expect("called next_key out of order")
.map(|key| key.into())
}
fn next_value<T: Deserialize>(&mut self) -> T {
self.use_item(|item| item.into())
.expect("called next_value out of order")
}
}
The key insight here is that ListMapAccess has to own a NonNull<List>, instead of a mutable reference. We provide an API that guarantees at compile time that the list can never be mutably borrowed more than once, because it only allows a mutable borrow to be created if self.item (which inherently borrows the list) is empty, and prevents the item from escaping ownership by the ListMapAccess.
The code here is a simplification of the real-world use case (it avoids a bunch of extra lifetimes and Result return types that aren't relevant to the problem at hand), but the real-world version is available here for those who are interested.