javapythonpysparkjvmpy4j

Import java package com.typesafe.config.impl.SimpleConfig using python with py4j

I can import all from java.* using the jvm from JavaGateway of py4j

for example:

jvm.java.util.ArrayList
jvm.java.lang.System
random = jvm.java.util.Random()
random.nextInt(3)

or in this form:

from py4j.java_gateway import java_import
java_import(jvm,'java.util.*')
random = jvm.Random()
random.nextInt(3)

I need to import:

com.typesafe.config.impl.SimpleConfig
com.typesafe.config.ConfigFactory
com.typesafe.config.Config

But it doesn't work, I tried these ways:

jvm.com.typesafe.config.impl.SimpleConfig
jvm.java.com.typesafe.config.impl.SimpleConfig
jvm.java.typesafe.config.impl.SimpleConfig
java_import(jvm,'com.typesafe.config.*')
java_import(jvm,'com.typesafe.config.impl.*')
jvm.SimpleConfig
jvm.impl.SimpleConfig

all returns: py4j.java_gateway.JavaPackage not py4j.java_gateway.JavaClass as needed

Solution

  • I had to download the JAR https://repo1.maven.org/maven2/com/typesafe/config/1.4.1/config-1.4.1.jar then add the JAR in the ("spark.jars", jar_path)