So I want to know if I can get the HQL query or the SQL query using the applicationId of a hive query that is running on YARN. I tried using yarn logs applicationid But it's showing the entire execution plan and container logs but I would like to know the attached HQL query to it
For. Eg ApplicationId -applicationId_00000_8282 Attached query -- Insert overwrite tablename partition whatever whatever
I don't think you can get the original SQL query statement from YARN log. What you could do is to get the Query Id from YARN and use the Query Id to find the original query statement from the Hive log. The Query Id is shown as the application tag in YARN log or YARN UI.
You can search for "Executing command" in hive log file. You should see something like
2024-09-24 17:01:15,870 INFO org.apache.hadoop.hive.ql.Driver: [HiveServer2-Background-Pool: Thread-34147]: Executing command(queryId=hive_20240924170115_e87016ed-57db-4bc7-a066-ee5472653a27): select * from foo where i=2 and city='aaa'
The log file should be in /var/log/hive directory. Please remember, if you have multiple Hive servers running in your cluster, to find the specific query statement, you might need to search every Hive server node in your cluster