I am trying to make a watchdog for a single-threaded program. The problem is, that we run some foreign so/dlls (the code is available) which means that we pass control there.
The idea is to recompile these with some callback to a sort of a cancellation routine.
Is it possible to let GCC call some callback functions in between of C-transactions or asm-transactions in this compiled foreign code?
What I'm about to suggest does not involve the compiler, but this sounds like a problem you can solve at runtime with POSIX signals or ptrace ...
With a signal you can interrupt the current context, similar to what would happen in kernel mode with an IRQ. You will have to worry about being "signal-safe" (example: your handler can't use malloc because it might interrupt malloc itself while its data structures are in an indeterminate state.)
With ptrace you can step through instructions in another process as if in a debugger.
Tread carefully, as these are difficult mechanisms to use correctly and it's very easy to shoot yourself in the foot.