pysparkknime

Knime BigData Local Environment & findspark

I'm just wondering: is possible to access the BigData Environment Node launched in Knime?

I have a node in a Knime workflow. From outside Knime, I wrote a Python script using findspark but ... it doesn't seem to detect it.

Any example or proposal to do that?

Thank you in advance

Solution

  • Hive is accessible while Spark and HDFS are not. You can connect to Hive via JDBC with port which is randomized upon node execution. Information about it should be in the first output port - DB Session.