pythonlinuxsqlitefull-text-searchfts5

Properly install sqlite3 with FTS5 support

I'm developing a Python tool which uses a sqlite3 virtual table with FTS5 (Full Text Search). I would like to know how to properly install from a tarball (or any other means) the needed requirements for my tool to work so I can pack them for portability.

Currently, I managed to install the latest release tarball of sqlite. However, when I execute:

python3 -c "import sqlite3; print(sqlite3.sqlite_version)"
# or
python2 -c "import sqlite3; print(sqlite3.sqlite_version)"

I get 3.11.0, while sqlite3 --version returns: 3.22.0 2018-01-22 18:45:57 0c55d179733b46d8d0ba4d88e01a25e10677046ee3da1d5b1581e86726f2alt1

The system version sqlite3 3.22 does support FTS5, as I do pragma compile_options; and get:

COMPILER=gcc-5.4.0 20160609
ENABLE_DBSTAT_VTAB
ENABLE_FTS4
**ENABLE_FTS5**
ENABLE_JSON1
ENABLE_RTREE
ENABLE_STMTVTAB
ENABLE_UNKNOWN_SQL_FUNCTION
HAVE_ISNAN
THREADSAFE=1

But, the python version, using this script returns this:

[(u'ENABLE_COLUMN_METADATA',), (u'ENABLE_DBSTAT_VTAB',), (u'ENABLE_FTS3',), (u'ENABLE_FTS3_PARENTHESIS',), (u'ENABLE_JSON1',), (u'ENABLE_LOAD_EXTENSION',), (u'ENABLE_RTREE',), (u'ENABLE_UNLOCK_NOTIFY',), (u'ENABLE_UPDATE_DELETE_LIMIT',), (u'HAVE_ISNAN',), (u'LIKE_DOESNT_MATCH_BLOBS',), (u'MAX_SCHEMA_RETRY=25',), (u'OMIT_LOOKASIDE',), (u'SECURE_DELETE',), (u'SOUNDEX',), (u'SYSTEM_MALLOC',), (u'TEMP_STORE=1',), (u'THREADSAFE=1',)]

Hence, my questions are:

  1. Is there any way I could make a linux portable package for my app with sqlite3 FTS5 support in both python and linux system?
  2. Is there any way to link the python module sqlite3 to an specific sqlite3 path?

I tried all of this in an Ubuntu 16.04 LTS, but I would like to work as well on CentOS 7.

Thank you very much in advance.

More details about the installation from the tarball that I did:

wget "https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release" -O sqlite.tar.gz
tar -xzvf sqlite.tar.gz
cd sqlite
./configure --enable-fts5
make
sudo make install
Solution

  • I think is a linking problem! I followed the same install steps with you and got the same results:

    $ python ./test.py 
[(u'ENABLE_COLUMN_METADATA',), (u'ENABLE_FTS3',), (u'ENABLE_RTREE',), (u'ENABLE_UNLOCK_NOTIFY',), (u'ENABLE_UPDATE_DELETE_LIMIT',), (u'MAX_SCHEMA_RETRY=25',), (u'OMIT_LOOKASIDE',), (u'SECURE_DELETE',), (u'SOUNDEX',), (u'SYSTEM_MALLOC',), (u'TEMP_STORE=1',), (u'THREADSAFE=1',)]
NO

    However, when you install something by configure/make/make install on Linux, it usually goes in /usr/local/lib. To make sure that python links on runtime against the correct .so I used LD_LIBRARY_PATH. In this case I got:

    $ LD_LIBRARY_PATH=/usr/local/lib python ./test.py 
[(u'COMPILER=gcc-4.8.5',), (u'ENABLE_FTS5',), (u'HAVE_ISNAN',), (u'TEMP_STORE=1',), (u'THREADSAFE=1',)]
YES

    Additionally, when installing libraries, you might have to update ldconfig. On my system (Ubuntu 14.04):

    $ sudo ldconfig
$ python ./test.py 
[(u'COMPILER=gcc-4.8.5',), (u'ENABLE_FTS5',), (u'HAVE_ISNAN',), (u'TEMP_STORE=1',), (u'THREADSAFE=1',)]
YES

    Notice that using LD_LIBRARY_PATH is not needed any more and python links against the correct lib. For this to happen you will need to have /usr/local/lib folder in your ld.so.conf somewhere... for me this is in:

    $ grep -ir local /etc/ld.so.conf.d/
/etc/ld.so.conf.d/libc.conf:/usr/local/lib