pythoncdeep-copygilpyobject

Is there a way to perform a deep copy of a PyObject without using the Python API (e.g. via C, Rust, etc)?

I was wondering if anyone knew of an implementation/library I could use to perform a deep copy of a PyObject without using the Python API.

I'd prefer something in C (as I currently use, and am somewhat familiar with CFFI), but anything (no matter the language - e.g. RUST), would be greatly appreciated.

The reason for this, is that I'm attempting to perform an analysis of Python variables (for a real-time Python debug library), but don't want to perform the analysis during the execution of the program being analyzed (as that would greatly impact program performance).

If I could analyze the variables post-execution (but before program termination), that would be tremendously helpful. In order to do that, I'd need to save the variables in some other thread (preferably a C program which doesn't require the GIL - so that the main Python program can continue execution uninterrupted).

I personally don't think there's anything out there, as I've looked already, but thought it might be worth a shot asking someone on Stack Overflow.

Thank you.

Solution

  • In C there is memcpy for making deep copies of structs and structs are the closest equivalent to an object in OOP that you can get. So if you can get the size of the Python object in memory and its memory location you can use memcpy() to copy it deeply (Deep copying array in C... malloc?, Making a deep copy of a struct...making a shallow copy of a struct). You can do this from within Python by either writing an additional module (https://docs.python.org/3/extending/extending.html) or mechanisms like cython (https://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html)

    However if the Python object contains pointers to substructures memcpy will not produce a deep copy (C++ deep copying with objects) In this case you have to allocate memory for the copy of the Python object and copy each substructure manually also

    https://agiledeveloper.com/articles/cloning072002.htm - Why Copying an Object is a terrible thing to do?

    Update :

    As stated in the comments using C memcpy is not an optimal solution because of substructures that are possibly implemented as pointers. So maybe try to use Python's copy module ( https://pymotw.com/2/copy/ ) or analyze its source code and adapt it for your needs