pythonc++destructorpybind11

PyBind11 destructor not invoked?


I have a c++ class wrapped with PyBind11. The issue is: when the Python script ends the c++ destructor is not being automatically invoked. This causes an untidy exit because networking resources need to be released by the destructor.

As a work-around it is necessary to explicitly delete the Python object, but I don't understand why!

Please could someone explain what is wrong here and how to get the destructor called automatically when the Python object is garbage collected?

Pybind11 binding code:

py::class_<pcs::Listener>(m, "listener")
    .def(py::init<const py::object &, const std::string &, const std::string &, const std::string &, const std::string &, const std::set<std::string> &, const std::string & , const bool & , const bool & >(), R"pbdoc(
    Monitors network traffic.

    When a desired data source is detected a client instance is connected to consume the data stream.

    Reconstructs data on receipt, like a jigsaw.  Makes requests to fill any gaps.  Verifies the data as sequential.

    Data is output by callback to Python.  Using the method specified in the constructor, which must accept a string argument.
)pbdoc");

In Python:

#Function to callback
def print_string(str):
    print("Python; " + str)

lstnr = listener(print_string, 'tcp://127.0.0.1:9001', clientCertPath, serverCertPath, proxyCertPath, desiredSources, 'time_series_data', enableCurve, enableVerbose)

#Run for a minute
cnt = 0
while cnt < 60:
    cnt += 1
    time.sleep(1)

#Need to call the destructor explicity for some reason    
del lstnr

Solution

  • So some years later I fixed this issue by enabling Python context manager with support by adding __enter__ and __exit__ method handling to my PyBind11 code:

    py::class_<pcs::Listener>(m, "listener")
    .def(py::init<const py::object &, const std::string &, const std::string &, const std::string &, const std::string &, const std::set<std::string> &, const std::string & , const bool & , const bool & >(), R"pbdoc(
        Monitors network traffic.
    
        When a desired data source is detected a client instance is connected to consume the data stream.
        
        Specify 'type' as 'string' or 'market_data' to facilitate appropriate handling of BarData or string messages.
    
        Reconstructs data on receipt, like a jigsaw.  Makes requests to fill any gaps.  Verifies the data as sequential.
    
        Data is output by callback to Python.  Using the method specified in the constructor, which must accept a string argument.
    )pbdoc")
    .def("__enter__", &pcs::Listener::enter, R"pbdoc(
        Python 'with' context manager support.
    )pbdoc")    
    .def("__exit__", &pcs::Listener::exit, R"pbdoc(
        Python 'with' context manager support.
    )pbdoc");
    

    Added corresponding functions to the C++ class, like so:

    //For Python 'with' context manager
    auto enter(){std::cout << "Context Manager: Enter" << std::endl; return py::cast(this); }//returns a pointer to this object for 'with'....'as' python functionality
    auto exit(py::handle type, py::handle value, py::handle traceback){ std::cout << "Context Manager: Exit: " << type << " " << value << " " << traceback <<  std::endl; }
    

    N.B.

    1. The returned pointer value from enter() is important to the as functionality in a with....as statement.

    2. The parameters passed to exit(py::handle type, py::handle value, py::handle traceback) are useful debugging info.

    Python usage:

    with listener(cb, endpoint, clientCertPath, serverCertPath, proxyCertPath, desiredSources, type, enableCurve, enableVerbose):
        cnt = 0
        while cnt < 10:
            cnt += 1
            time.sleep(1)
    

    The Python context manager now calls the pcs::Listener::exit on the C++ object, which should release the networking resources.