How is the default tokenizer 'porter' in fts3 module registered.
one way to register user defined tokenizers is fts.register_tokenizer() but what are its arguments?
since porter being a in built tokenizer does it even need to be registered?
If your Python has the Porter tokenizer compiled in, you do not need to register it.
To register a user-defined tokenizer, you have to call the fts3_tokenizer() SQL function with a pointer to a C structure containing C function pointers; this cannot be easily done from Python.