pythonc++python-3.xpython-importextension-modules

C++ Python module import error: "undefined symbol: Py_InitModule3" ( Py_InitModule () )


I am just starting an attempt to write my first Python extension module in C and are using instructions provided at https://www.tutorialspoint.com/python/python_further_extensions.htm

I am on Linux Mint 18.1, using Python 3.6.1 in its virtualenv version.

As first step I have compiled a very minimalistic version of the Python module I plan to write.

Here my C-code:

include <Python.h>

static PyObject* uniqueCombinations(PyObject* self)
{
    return Py_BuildValue("s", "uniqueCombinations() return value (is of type 'string')");
}

static char uniqueCombinations_docs[] =
    "usage: uniqueCombinations(lstSortableItems, comboSize)\n";

static PyMethodDef uniqueCombinations_funcs[] = {
    {"uniqueCombinations", (PyCFunction)uniqueCombinations, 
     METH_NOARGS, uniqueCombinations_docs},
    {NULL}
};

void inituniqueCombinations(void)
{
    Py_InitModule3("uniqueCombinations", uniqueCombinations_funcs,
                   "Extension module uniqueCombinations v. 0.01");
}

and here the code of the setup.py file I have used to compile the C-code to uniqueCombinations.cpython-36m-x86_64-linux-gnu.so:

from distutils.core import setup, Extension

setup(name='uniqueCombinations', version='0.01',  \
      ext_modules=[Extension('uniqueCombinations', ['uniqueCombinations_pythonCmodule_v-0.01_Cg.c'])])

To my surprise the module doesn't load properly on import:

>>> import uniqueCombinations
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: uniqueCombinations.cpython-36m-x86_64-linux-gnu.so: undefined symbol: Py_InitModule3

Why am I getting this error?

And what should I do to be able to load the compiled module properly?

Here the build protocol with a warning about implicit declaration of Py_InitModule3 :

$ python uniqueCombinations_pythonCmodule_v-0.01_setup.py_Cg.py build
running build
running build_ext
building 'uniqueCombinations' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.6m -c uniqueCombinations_pythonCmodule_v-0.01_Cg.c -o build/temp.linux-x86_64-3.6/uniqueCombinations_pythonCmodule_v-0.01_Cg.o
uniqueCombinations_pythonCmodule_v-0.01_Cg.c: In function ‘inituniqueCombinations’:
uniqueCombinations_pythonCmodule_v-0.01_Cg.c:19:5: warning: implicit declaration of function ‘Py_InitModule3’ [-Wimplicit-function-declaration]
     Py_InitModule3("uniqueCombinations", uniqueCombinations_funcs,
     ^
creating build/lib.linux-x86_64-3.6
gcc -pthread -shared build/temp.linux-x86_64-3.6/uniqueCombinations_pythonCmodule_v-0.01_Cg.o -o build/lib.linux-x86_64-3.6/uniqueCombinations.cpython-36m-x86_64-linux-gnu.so

Solution

  • The Py_InitModule (Py_InitModule3) is no longer used. It is necessary to create a PyModuleDef structure instead and then pass a reference to it to PyModule_Create.

    Below the C-code from which created module doesn't throw an error when imported:

    #include <Python.h>
    
    static PyObject* uniqueCombinations(PyObject* self)
    {
        return Py_BuildValue("s", "uniqueCombinations() return value (is of type 'string')");
    }
    
    static char uniqueCombinations_docs[] =
        "usage: uniqueCombinations(lstSortableItems, comboSize)\n";
    
    /* deprecated: 
    static PyMethodDef uniqueCombinations_funcs[] = {
        {"uniqueCombinations", (PyCFunction)uniqueCombinations, 
         METH_NOARGS, uniqueCombinations_docs},
        {NULL}
    };
    use instead of the above: */
    
    static PyMethodDef module_methods[] = {
        {"uniqueCombinations", (PyCFunction) uniqueCombinations, 
         METH_NOARGS, uniqueCombinations_docs},
        {NULL}
    };
    
    
    /* deprecated : 
    PyMODINIT_FUNC init_uniqueCombinations(void)
    {
        Py_InitModule3("uniqueCombinations", uniqueCombinations_funcs,
                       "Extension module uniqueCombinations v. 0.01");
    }
    */
    
    static struct PyModuleDef Combinations =
    {
        PyModuleDef_HEAD_INIT,
        "Combinations", /* name of module */
        "usage: Combinations.uniqueCombinations(lstSortableItems, comboSize)\n", /* module documentation, may be NULL */
        -1,   /* size of per-interpreter state of the module, or -1 if the module keeps state in global variables. */
        module_methods
    };
    
    PyMODINIT_FUNC PyInit_Combinations(void)
    {
        return PyModule_Create(&Combinations);
    }
    

    Here the build protocol:

    $ python uniqueCombinations_pythonCmodule_v-0.01_setup.py_Cg.py build
    running build
    running build_ext
    building 'Combinations' extension
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.6m -c uniqueCombinations_pythonCmodule_v-0.01_Cg.c -o build/temp.linux-x86_64-3.6/uniqueCombinations_pythonCmodule_v-0.01_Cg.o
    gcc -pthread -shared build/temp.linux-x86_64-3.6/uniqueCombinations_pythonCmodule_v-0.01_Cg.o -o build/lib.linux-x86_64-3.6/Combinations.cpython-36m-x86_64-linux-gnu.so
    

    and the interactive Python console session with successful import:

    >>> import Combinations
    >>> dir(Combinations)
    ['__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'uniqueCombinations']
    >>> Combinations.__doc__
    'usage: Combinations.uniqueCombinations(lstSortableItems, comboSize)\n'
    >>> Combinations.uniqueCombinations()
    "uniqueCombinations() return value (is of type 'string')"
    >>>