pythonrustpyo3

Interoperability between two crates with `pyo3` bindings


I have two Rust crates lib1 and lib2, for which I use pyo3 to generate bindings. I import the crate lib1 in lib2.

In a separate crape lib1-py, I create a python class MyClass, and in lib2, I try to use MyClass as a parameter for a Python function, using :

fn foo(param: PyRef<MyClass>)

I separately use maturin to build both lib1-py and lib2 in the same environment.

When I try to run the python code:

import lib1
import lib2

param = lib1.MyClass()
lib2.foo(param)

I obtain the error TypeError: argument 'param': 'MyClass' object cannot be converted to ‘MyClass’.

I suppose that Rust's crate system tries to be safe by treating types from different crates as separate entities, even if they are identical. How could I solve such an issue?


Solution

  • In general, this is impossible. Rust does not even guarantee MyClass will have the same memory layouts in both builds.

    It is possible to design lib1-py to allow this, though. For example, if the entirety of MyClass is #[repr(C)] (that is, it and all of its fields are, recursively), you can take a Bound<PyAny> and convert it to Bound<MyClass> with into_ptr() and from_owned_ptr(). Although that is a dangerous method; if something will someday become not #[repr(C)] your code will become unsound.

    A probably better method (and what e.g. Polars does) is to have a method to serialize MyClass to some format, accessible from Python, and a corresponding method to deserialize (which doesn't need to be accessible from Python). Then you take Bound<PyAny> and call this method on it, and deserialize the result.