cythonshared-ptrmake-shared

How to use shared_ptr and make_shared with arrays?


I want to use a C++ shared_ptr as a replacement for raw C pointers. As a simple example the following code seems to work as intended:

from libcpp.memory cimport shared_ptr, allocator

cdef shared_ptr[double] spd 
cdef allocator[double] allo
spd.reset(allo.allocate(13))

The size is chosen as 13 here, but in general is not know at compile time. I'm not sure if this is correct, but I haven't had any errors (no memory leaks and segfaults yet). I'm curious if there is a more optimal solution with make_shared.

But I can't use C++11 arrays because Cython doesn't allow literals as templates, e.g. something like

cdef shared_ptr[array[double]] spd = make_shared[array[double,13]]()

and "normal" arrays which are supposed to work with C++20 compiler (e.g. gcc 10) are causing problems in one way or another:

# Cython error "Expected an identifier or literal"
cdef shared_ptr[double[]] spd = make_shared[double[]](3)    

# Can't get ptr to work
ctypedef double[] darr
cdef shared_ptr[darr] spd = make_shared[darr](13)
cdef double* ptr = spd.get()    # or spd.get()[0] or <double*> spd.get()[0] or ...

Is the allocator solution the correct and best one or is there another way how to do it?


Solution

  • Here is what I'm going with

    cdef extern from *:
        """
        template <typename T>
        struct Ptr_deleter{
            size_t nn;
            void (*free_ptr)(T*, size_t);
            Ptr_deleter(size_t nIn, void (*free_ptrIn)(T*, size_t)){
                this->nn = nIn;
                this->free_ptr = free_ptrIn;
            };
            void operator()(T* ptr){
                free_ptr(ptr, nn);
            };
        };
        template <typename T>
        std::shared_ptr<T> ptr_to_sptr (T* ptr, size_t nn, void (*free_ptr)(T*, size_t)) {
            Ptr_deleter dltr(nn,  free_ptr);
            std::shared_ptr<T> sp (ptr, dltr);
            return sp;
        };
        """
        shared_ptr[double] d_ptr_to_sptr "ptr_to_sptr"(double* ptr, size_t nn, void (*free_ptr)(double*, size_t) nogil) nogil
    
    cdef void free_d_ptr(double* ptr, size_t nn) nogil:
        free(ptr)
    
    cdef shared_ptr[double] sp_d_empty(size_t nn) nogil:
        return d_ptr_to_sptr(<double*> nullCheckMalloc(nn*sizeof(double)), nn, &free_d_ptr)
    

    My understanding is that the "right" way to handle malloced arrays is to use a custom deleter like I did. I personally prefer sticking with somewhat-raw C pointers (double* instead of double[] or something), since it's more natural in Cython and my projects.

    I think it's reasonably easy to see how to change free_ptr for more complicated data types. For simple data types it could be done in less lines and less convoluted, but I wanted to have the same base.

    I like my solution in the regard that I can just "wrap" existing Cython/C code raw pointers in a shared_ptr.

    When working with C++ (especially newer standards like C++20) I think verbatim code is pretty often necessary. But I've intentionally defined free_d_ptr in Cython, so it's easy to use existing Cython code to handle the actual work done to free/clear/whatever the array.

    I didn't get C++11 std::arrays to work, and it's apparently not "properly" possible in Cython in general (see Interfacing C++11 array with Cython).

    I didn't get double[] or similar to work either (is possible in C++20), but with verbatim C++ code I think this should be doable in Cython. I prefer more C-like pointers/arrays anyway as I said.