I am trying to perform a simple query on Hive - Spark engine:
SELECT count(*) FROM classification_output GROUP BY model_id;
I keep receiving the below error:
Error: Error while compiling statement: FAILED: SemanticException Failed to get a spark session: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create spark client. (state=42000,code=40000)
Only ONT time it worked after 12 minutes
The number of records on the table are around ~7K, I suspect a configuration issue, but I am not sure which parameter.
I tried to:
Note:
Using:
SELECT count(*) FROM classification_output;
With no group by....
works just fine and return the below:
+-------+
| _c0 |
+-------+
| 7164 |
+-------+
1 row selected (0.092 seconds)
After spending some time around this issue, I managed to find the solution. It has nothing to do with the query. There was another SparkClient process running, once I stopped it and executed the query it works fine.