regexhivemapreducehiveqlrlike

How to write fuzzy multiple substring matching when using RLIKE in Hive


For example:

df.select('category').show()

+---------------------------+
|                   category|
+---------------------------+
|            money,insurance|
|            life, housework|
|           game,FPS,network|
|            game,fight,jump|
|                      hotel|
|                 trip,hotel|
|                       null|

I want to use RLIKE to write a regex expression to fuzzy match one of substrings list, ['money', 'life'].

-- This is an exact match
SELECT * 
FROM tb_name
WHERE col_name RLIKE '(money|life)'

-- This is a fuzzy match
SELECT * 
FROM tb_name
WHERE col_name RLIKE '*.(money|life)'

BUT there is error in ast tree in the fuzzy match code snippet.

06-11 16:59:17-fatal filter ast tree

(TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TAB tb_name))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR "hdfs://XXXX/XX")) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_WHERE (RLIKE (TOK_TABLE_OR_COL col_name ) '*.(money|life)')) (TOK_LIMIT 2000)))

06-11 16:59:17-fatal Filter feature: .TOK_TAB \S tdw_inter_db.*|.TOK_(CUBE|ROLLUP) .

So I can't see anything wrong with the fuzzy match code snippet.
So could anyone help me?
Thanks in advances.


Solution

  • '(?i)money|life' regexp will match strings containing any of money, life, case insensitive - (?i)