pythondatabrickskeywordsql-parser

Does sqlparse.token.keyword recognize Top as a keyword?


I'm trying this code where I want to convert T-SQL to databricks-SQL and I'm trying to change only the uncommon by first defining the common keywords and check it against the parsed list. I want the code to go to next step of transformation when it encounters the keyword top as its not defined in databricks. but that is not happening.

This is my code

parsed = sqlparse.parse(sql_query)[0]
tokens = TokenList(parsed.tokens).flatten()

common_keywords = ["SELECT","FROM","DISTINCT","WHERE","GROUP BY","ORDER BY","AS","JOIN","VALUES","INSERT INTO","UPDATE","DELETE FROM","SET","COUNT","AVG","MIN","MAX","SUM"]
flag=True
for token in tokens:
    if token.ttype==Keyword:
        if token.value.upper() not in common_keywords:
            flag=False
            break
if flag:
    return sql_query

later I'm writing the code for replacing top with limit at the end. but the o/p of this is the query itself instead of break.


Solution

  • So it turns out "sqlparse.token.Keyword" only recognizes DDL and DML keywords and not DQL keywords. so it wasn't even recognizing TOP to be a keyword. I just added another condition for making flag false (I'm also checking for char "@" there)

    for i in sql_query:
            if "TOP" in i or "@" in i:
                flag=False
    

    I'll leave a GitHub link which has list of all word, functions and character's recognized by "sqlparse.token.Keyword" https://github.com/andialbrecht/sqlparse/blob/master/sqlparse/keywords.py