sqlhadoopimpala

Bash Variable inside sql query. CAUSED BY: Exception: Syntax error


I have a shell script where insert data from a file into hive table via impala.

#declare variables
YEAR=$(date +"%Y")
MONTH=$(date +"%-m")
DAY=$(date +"%-d") 
.....

$IMPALA_CONNECTION -q "LOAD DATA INPATH '$HDFS_DIR/text_file.txt' INTO TABLE db.tbl_stg;
INSERT INTO db.tbl PARTITION(year, month, day) SELECT rest, of, sql, $YEAR, $MONTH, $DAY FROM 
db.tbl_stg;

ERROR: ParseException: Syntax error in line 1: ..., 2023, 5, 9  FROM db.tbl_stg... ^ For some reason it does not like $DAY variable because if I hardcode 9, it works perfectly.

This is if I echo the query:

impala-shell -k --ssl -i pserver:21000 --ca_cert=path_to_cert.pem -q 
LOAD DATA INPATH '$hdfs_path_here/impala_query_hist.txt' INTO TABLE 
db.tbl_stg;

INSERT INTO db.tbl PARTITION(year, month, day) SELECT ......, ...., 
memory_accrual, ddl_type, rows_inserted, hdfs_bytes_written, 2023, 5, 9  
FROM db.tbl_stg;

Solution

  • Thanks everyone for helpful comments. There was an extra space at the end of the variable DAY=$(date +"%-d") . You can see it if you highlight that light in my original question post. After removing the extra space everything is worked