pythonlibrariesdynet

How to check what binary library file does python link to?

Some time ago, I installed dynet for Python and it worked fine. I installed it automatically with pip install git+https://github.com/clab/dynet#egg=dynet.

Then, I wanted to add MKL support. So I installed dynet again manually, using the instructions here: http://dynet.readthedocs.io/en/latest/python.html#manual-installation, passing a -DMKL parameter to cmake.

I want to make sure that I did everything correctly and that Python indeed takes the new installation of dynet. How can I check this?

In general, when I have a library that is built in C++ and linked from Python, how can I check, from within Python, the details of the installed library, and particularly, what binary file does Python link to?

Solution

  • This is tricky. There is no standard way, you have to rely on information provided by the package (which many don't do) and on implementation details for the specific library.

    Python libraries often expose their version number in a __version__ attribute. This attribute is not standardised, it is at most a convention. The dynet library does have this attribute but it doesn't list the patch-level version, only the major and minor versions:

    >>> import dynet
>>> dynet.__version__
'2.0'

    Because you installed the library either directly from the version control system (with git+https://...) or manually from source, you can't use pip freeze or pkg_resources.get_distribution() to inspect the version either:

    $ pip freeze | grep -i dynet
dyNET==0.0.0

    Compilation flags are not stored anywhere, normally. Python exposes it's own compilation-time info in structures in the sys module and the sysconfig module, but there is no such facility for extension modules. Unless the extension module explicitly includes such info in the API, you are out of luck there.

    At best, you can try to locate the actual dynamic module loaded. Many projects use a wrapper Python module, which indirectly loads the actual extension module, confusing matters. In this case, importing dynet gives you the dynet.py file:

    >>> dynet.__file__
'/.../lib/python3.6/site-packages/dynet.py'

    This file was generated from the dynet.py.in file in the project source code. It imports the dynamic module as _dynet:

    from _dynet import *

    So you can still at least find the location of the dynamic object loaded with:

    >>> import _dynet
>>> _dynet.__file__
'/.../lib/python3.6/site-packages/_dynet.cpython-36m-darwin.so'

    You can further check what that dynamic library links to if you want to check they are the right versions; how you do this is platform dependent: