pythonpackagepythran

import pythran module in a python packages with PythranExtension


I try to make a python package using the excellent pythran.

With this file structure

$ tree ../proj
../proj
├── ccompile.py
├── Makefile
├── proj
│   ├── __init__.py
│   └── lib.py
└── setup.py

My pythran stuff is in ccompile.py:

$ cat ccompile.py
# pythran export get_fast_results(int[], int[][], (int, int):int dict)
def get_fast_results(mylist_of_int, myarray, dict_with_int_tuples_keys):
    # do stuff update dict_with_int_tuples_keys
    return dict_with_int_tuples_keys

The setup.py call PythranExtension:

$ cat setup.py
from setuptools import setup
from pythran.dist import PythranExtension

setup(name='proj',
      version=0.42,
      packages=['proj'],
      ext_modules=[PythranExtension("ccompile", ["ccompile.py"])],
      # ext_modules=[PythranExtension("proj.ccompile", ["proj/ccompile.py"])],
      zip_safe=False)

Everything work well python setup.py install compile ccompile.py and install the .so at the root of the egg folder::

$ tree
~/venv/lib/python2.7/site-packages/proj-0.42-py2.7-linux-x86_64.egg/
├── ccompile.py
├── ccompile.pyc
├── ccompile.so
├── EGG-INFO
│   └── ...
└── proj
    ├── __init__.py
    ├── __init__.pyc
    ├── lib.py
    └── lib.pyc

The problem is that I have to manually mv ccompile.* from site-packages/proj-0.42-py2.7-linux-x86_64.egg/ to site-packages/proj-0.42-py2.7-linux-x86_64.egg/proj/ to be able to import pythran function from proj package::

/tmp$ python -c 'from proj.ccompile import get_fast_results as gfr; print gfr.__doc__'
Supported prototypes:
    - get_fast_results(int[], int[][], (int, int):int dict)
    - get_fast_results(int[], int[][].T, (int, int):int dict)
 make it fast
/tmp$

If I use the syntax from distutils Extension ( Cf. https://docs.python.org/2/distutils/examples.html#single-extension-module ) e.g. ext_modules=[PythranExtension("proj.ccompile", ["proj/ccompile.py"])], in setup.py (moving ccompile in proj) I get an error when pythran compile it:

$ tree ../proj
../proj
├── Makefile
├── proj
│   ├── ccompile.cpp
│   ├── ccompile.py
│   ├── __init__.py
│   └── lib.py
└── setup.py

with::

$ more setup.py 
from setuptools import setup
from pythran.dist import PythranExtension

setup(...
      ext_modules=[PythranExtension("proj.ccompile", ["proj/ccompile.py"])],
      zip_safe=False)

I get::

$ python setup.py install
running install
...
running egg_info
...
copying proj/__init__.py -> build/lib.linux-x86_64-2.7/proj
copying proj/ccompile.py -> build/lib.linux-x86_64-2.7/proj
copying proj/lib.py -> build/lib.linux-x86_64-2.7/proj
running build_ext
building 'proj.ccompile' extension
C compiler: x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/proj
compile options: '-DUSE_GMP -DENABLE_PYTHON_MODULE -I~/venv/local/lib/python2.7/site-packages/pythran -I/usr/include/python2.7 -c'
extra options: '-std=c++11 -fno-math-errno -w'
x86_64-linux-gnu-gcc: proj/ccompile.cpp
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
proj/ccompile.cpp:35:25: error: expected ‘{’ before ‘.’ token
 namespace __pythran_proj.ccompile
                         ^
proj/ccompile.cpp:35:25: error: expected unqualified-id before ‘.’ token
proj/ccompile.cpp:610:13: error: expected ‘}’ at end of input
             }
             ^
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
proj/ccompile.cpp:35:25: error: expected ‘{’ before ‘.’ token
 namespace __pythran_proj.ccompile
                         ^
proj/ccompile.cpp:35:25: error: expected unqualified-id before ‘.’ token
proj/ccompile.cpp:610:13: error: expected ‘}’ at end of input
             }
             ^
error: Command "x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DUSE_GMP -DENABLE_PYTHON_MODULE -I~/venv/local/lib/python2.7/site-packages/pythran -I/usr/include/python2.7 -c proj/ccompile.cpp -o build/temp.linux-x86_64-2.7/proj/ccompile.o -std=c++11
-fno-math-errno -w" failed with exit status 1
$

Solution

  • This was a pythran issue. It has been fixed in commit 18aefe0e2383.