Can anyone help me with starting spark thrift server? I am running my script in standalone mode and I want to fetch data in my business intelligence tool.
In order to do that I need to start thrift server. I tried running shell script:
$SPARK_HOME/sbin/start-thriftserver.sh
but I get an error:
error "ps unknown option --0"
The sbin/ scripts work fine under linux, but they are not prepared to run in Windows (see Failed to start master for Spark in Windows). But spark thriftserver can be start in foreground independent from OS using command:
java -cp conf/:jars/* org.apache.spark.deploy.SparkSubmit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 spark-internal
Then you can test connection using beeline supplied with spark:
$ ./bin/beeline -u jdbc:hive2://localhost:10000
Connecting to jdbc:hive2://localhost:10000
16/11/22 13:09:57 INFO Utils: Supplied authorities: localhost:10000
16/11/22 13:09:57 INFO Utils: Resolved authority: localhost:10000
16/11/22 13:09:57 INFO HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://localhost:10000
Connected to: Spark SQL (version 2.0.1)
Driver: Hive JDBC (version 1.2.1.spark2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.2.1.spark2 by Apache Hive
0: jdbc:hive2://localhost:10000> show databases;
+---------------+--+
| databaseName |
+---------------+--+
| default |
| elo |
+---------------+--+
2 rows selected (0,26 seconds)