I'm trying to create a custom analyzer, following the syntax from https://crate.io/docs/crate/reference/en/latest/sql/statements/create-analyzer.html however when I attempt to create the following :
create analyzer FullAddressAnalyzer ([TOKENIZER ngram with (min_gram = 2, max_gram =10)])
I get the SQLActionException[SQLParseException: tokenizer name 'ngram' is reserved] error, this baffles me as the documentation explains that you can use parameters for ngram but this doesn't seem to make any sense
Seems like the CrateDB SQL reference documentation is not correct for this case.
When creating a custom analyzer with a parameterized tokenizer, one must use a custom name for the tokenizer while defining the type of the tokenizer. Example:
create analyzer full_address_analyzer (TOKENIZER my_ngram with (type = ngram, min_gram = 2, max_gram =10)])