apache-sparkpysparkapache-spark-sql

'ilike' keyword not working with spark SQL

I am learning apache-spark. And what seems confusing is,

this command works

results = spark.sql("select * from df_sql_view where name LIKE '%i%'")

but this throws an error

results = spark.sql("select * from df_sql_view where name ILIKE '%i%'")

ParseException: 
mismatched input 'ILIKE' expecting {<EOF>, ';'}(line 1, pos 37)

== SQL ==
select * from df_sql_view where name ILIKE '%i%'
-------------------------------------^^^

Could someone please help me understand why and how to resolve it. Thank you.

Solution

  • Spark SQL only support like and rlike. So you'd want to use rlike with proper regex to replace ilike