Pleasantly surprised at the ease of integrating Rust modules into a calling Python app using PyO3.
But the next thing I want to understand is whether it's possible to exchange data between a potentially long-running Rust module and the Python code. A typical case would be where Python is handling the GUI (e.g. PyQt), and a user may want to terminate this long-running Rust-module midstream. So hopefully a graceful interrupt mechanism which the Rust module could detect. "Graceful" would tend to mean not using SIGINT, I assume...
But there's also a need for Rust to be able to send out signals in the other direction: a typical case would be a progress indicator to be updated in the GUI. But in another scenario actual significant objects might need to "spun off" from the Rust code and "caught" by the Python code.
Is any of this possible? I can think of an incredibly clunky mechanism: files. If the Python code wants to interrupt, it creates a particular file on disk. The lengthy Rust code is constantly checking to see whether such a file exists, and what the instruction is. Such file mechanism could also be used for data flowing in the other direction.
But I'm hoping there's a better method than this. I've done some searching but not found anything very obvious. An intriguing comment said "Maybe use crossbeam-channel to allow communication between Python and Rust, enabling Python to send a signal/Interrupt to the Rust program?". I had a look at crossbeam-channel, but couldn't see anything addressing this sort of scenario.
Naturally I also looked at PyO3. There doesn't seem to be anything about communication between the respective threads/processes (NB I'm not yet clear whether a called Rust module is in fact running in a different process to the calling Python, but I assume so).
Later
In fact, according to my experiments, it turns out that, unless you arrange things otherwise, e.g. by using Python subprocess or multiprocessing, etc., in fact a Rust-to-Python module compiled using maturin develop runs in the same process as the calling Python code.
The significance of this for this question is that it may be possible to use inter-thread-communication, potentially. zeromq, which uses sockets, seems adequate for me to be getting on with (and it is obviously a form of inter-thread communication), and I have no idea how such "superior" within-process inter-thread comms might work. Maybe a suitable expert might have an idea.
It is also intriguing to wonder how the GIL fits into this, when you have a PyO3 (maturin) module running in one thread, and some Python code running in another thread of the same process...
Looked at a lot of different possibilities.
Best solution seems to be explained here, option 3, zeromq.
Managed to get simple messages between Python and Rust in minutes (on W10).
I've got a general impression that sockets are slower than pipes. Struggled with various possible solutions, some only for *nix. If anyone can find and give an example of a pipes solution ("named pipes") for Windows 10, that'd be great.
Two important caveats
maturin develop this failed. However I have not checked that this is definitely the cause: it may be that another dependency is the problem. Consequently I have put my zmq functionality in another local crate (there is no problem useing this from the PyO3-enabled crate).