pythoncythoncythonize

Cython doesn't determine memoryview dimension change in sliced argument


I'm working with memoryviews of a fused type in a Cython application and have run into an issue where I expect Cython to know that a sliced 2D memoryview is a 1D memoryview, but can only get it to compile in 1 of 3 different ways.

# cython: language_level=3, boundscheck=False, cdivision=True, wraparound=False, initializedcheck=False, nonecheck=False
cimport cython


cdef void _subfunc(cython.floating[:] arr1, cython.floating[:] out1, cython.floating[:] out2):
    out1[0] = arr1[0] ** 2
    out2[0] = arr1[0] + 2


cdef void _func1(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, out1[:, 0], out1[:, 1])


cdef void _func2(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, <cython.floating[:]>out1[:, 0], <cython.floating[:]>out1[:, 1])


cdef void _func3(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    cdef cython.floating[:] o1, o2
    o1 = out1[:, 0]
    o2 = out1[:, 1]
    _subfunc(arr1, o1, o2)

When compiled with cythonize test.pyx I get the following errors:

Error compiling Cython file:
------------------------------------------------------------
...
cdef void _subfunc(cython.floating[:] arr1, cython.floating[:] out1, cython.floating[:] out2):
    out1[0] = arr1[0] ** 2
    out2[0] = arr1[0] + 2

cdef void _func1(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, out1[:, 0], out1[:, 1])
            ^
------------------------------------------------------------

test.pyx:10:12: no suitable method found

Error compiling Cython file:
------------------------------------------------------------
...
cdef void _subfunc(cython.floating[:] arr1, cython.floating[:] out1, cython.floating[:] out2):
    out1[0] = arr1[0] ** 2
    out2[0] = arr1[0] + 2

cdef void _func1(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, out1[:, 0], out1[:, 1])
            ^
------------------------------------------------------------

test.pyx:10:12: no suitable method found

Error compiling Cython file:
------------------------------------------------------------
...

cdef void _func1(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, out1[:, 0], out1[:, 1])

cdef void _func2(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, <cython.floating[:]>out1[:, 0], <cython.floating[:]>out1[:, 1])
                                           ^
------------------------------------------------------------

test.pyx:13:43: Can only create cython.array from pointer or array

Error compiling Cython file:
------------------------------------------------------------
...

cdef void _func1(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, out1[:, 0], out1[:, 1])

cdef void _func2(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, <cython.floating[:]>out1[:, 0], <cython.floating[:]>out1[:, 1])
                                                                           ^
------------------------------------------------------------

test.pyx:13:75: Can only create cython.array from pointer or array

Error compiling Cython file:
------------------------------------------------------------
...

cdef void _func1(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, out1[:, 0], out1[:, 1])

cdef void _func2(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, <cython.floating[:]>out1[:, 0], <cython.floating[:]>out1[:, 1])
                                           ^
------------------------------------------------------------

test.pyx:13:43: Can only create cython.array from pointer or array

Error compiling Cython file:
------------------------------------------------------------
...

cdef void _func1(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, out1[:, 0], out1[:, 1])

cdef void _func2(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, <cython.floating[:]>out1[:, 0], <cython.floating[:]>out1[:, 1])
                                                                           ^
------------------------------------------------------------

test.pyx:13:75: Can only create cython.array from pointer or array

Error compiling Cython file:
------------------------------------------------------------
...
cdef void _subfunc(cython.floating[:] arr1, cython.floating[:] out1, cython.floating[:] out2):
    out1[0] = arr1[0] ** 2
    out2[0] = arr1[0] + 2

cdef void _func1(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, out1[:, 0], out1[:, 1])
    ^
------------------------------------------------------------

test.pyx:10:4: Invalid use of fused types, type cannot be specialized

Error compiling Cython file:
------------------------------------------------------------
...
cdef void _subfunc(cython.floating[:] arr1, cython.floating[:] out1, cython.floating[:] out2):
    out1[0] = arr1[0] ** 2
    out2[0] = arr1[0] + 2

cdef void _func1(cython.floating[:] arr1, cython.floating[:, ::1] out1):
    _subfunc(arr1, out1[:, 0], out1[:, 1])
    ^
------------------------------------------------------------

test.pyx:10:4: Invalid use of fused types, type cannot be specialized

As you can see, only func3 with its explicit variables "o1" and "o2" satisfy the compilation. This is unexpected but also annoying since it is a lot more code. Is this just the way this has to be? Or am I doing something wrong?

Edit: It fails with the same errors with Cython 0.29.x and Cython 3.x.


Solution

  • Explicitly specifying cython.floating in the _subfunc call makes _func1 compiling. Have no idea why, I just give it a try and it works, hope some expert could add the internal mechanisms.

    cdef void _func1(cython.floating[:] arr1, cython.floating[:, ::1] out1):
        _subfunc[cython.floating](arr1, out1[:, 0], out1[:, 1])
    

    https://cython.readthedocs.io/en/latest/src/userguide/fusedtypes.html#selecting-specializations

    Oh, cython-3.0.0b3 is used.