I try to run Python script in GraalVm and it fails on
import requests
And it seems GraalVM issue, as the script work with usual python3
.
Java code:
import org.graalvm.polyglot.Context;
import org.graalvm.polyglot.Source;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.nio.file.Paths;
public class RunPython4 {
public static String PYTHON = "python";
// venv/**/* must be included into resources in pom.xml
private static String VENV_EXECUTABLE = RunPython4.class.getClassLoader().getResource(Paths.get("venv", "bin", "graalpython").toString()).getPath();
private static String SOURCE_FILE_NAME = "health.py";
public static void log(String s){
System.out.println(s);
}
public static void main(String[] args) {
log("Hello Java!");
log(System.getProperty("java.version"));
log(System.getProperty("java.runtime.version"));
String pyFilename = "./health.py";
try (Context context = Context.newBuilder("python").
allowAllAccess(true).
option("python.ForceImportSite", "true").
option("python.Executable", VENV_EXECUTABLE).
build();) {
context.eval(PYTHON, "print('Hello Python!')");
context.eval(PYTHON, "import sys; print(sys.version)");
//4
try(BufferedReader br = new BufferedReader(new FileReader(pyFilename))) {
int i = 0;
for(String line; (line = br.readLine()) != null; ) {
i++;
log(""+i+": "+line);
context.eval("python", line);
}
}catch ( IOException e){
log("IOException "+e);
}
Output:
Hello Java!
11.0.11
11.0.11+8-jvmci-21.1-b05
Hello Python!
3.8.5 (Fri Jun 25 17:55:09 CST 2021)
[Graal, GraalVM CE, Java 11.0.11]
1: # Checking subgraph health v1.1
2: # https://thegraph.com/docs/deploy-a-subgraph#checking-subgraph-health
3:
4: import requests
Exception in thread "main" Exception: You need either charset_normalizer or chardet installed
at org.graalvm.sdk/org.graalvm.polyglot.Context.eval(Context.java:379)
at RunPython4.main(RunPython4.java:54)
Process finished with exit code 1
I searched deeper, "You need either charset_normalizer or chardet installed"
is string from
https://github.com/psf/requests/blob/master/requests/__init__.py#L57-L85
def check_compatibility(urllib3_version, chardet_version, charset_normalizer_version):
urllib3_version = urllib3_version.split('.')
assert urllib3_version != ['dev'] # Verify urllib3 isn't installed from git.
# Sometimes, urllib3 only reports its version as 16.1.
if len(urllib3_version) == 2:
urllib3_version.append('0')
# Check urllib3 for compatibility.
major, minor, patch = urllib3_version # noqa: F811
major, minor, patch = int(major), int(minor), int(patch)
# urllib3 >= 1.21.1, <= 1.26
assert major == 1
assert minor >= 21
assert minor <= 26
# Check charset_normalizer for compatibility.
if chardet_version:
major, minor, patch = chardet_version.split('.')[:3]
major, minor, patch = int(major), int(minor), int(patch)
# chardet_version >= 3.0.2, < 5.0.0
assert (3, 0, 2) <= (major, minor, patch) < (5, 0, 0)
elif charset_normalizer_version:
major, minor, patch = charset_normalizer_version.split('.')[:3]
major, minor, patch = int(major), int(minor), int(patch)
# charset_normalizer >= 2.0.0 < 3.0.0
assert (2, 0, 0) <= (major, minor, patch) < (3, 0, 0)
else:
raise Exception("You need either charset_normalizer or chardet installed")
So it is clear that requests
init fails over charset_normalizer_version
that is likely undefined
https://github.com/psf/requests/blob/master/requests/__init__.py#L47
try:
from charset_normalizer import __version__ as charset_normalizer_version
except ImportError:
charset_normalizer_version = None
The requests
package is present of course
The problem was, that I was installing requests
using system python and not with graalpython -m ginstall install requests
Some packages are complex to compile, and to be worry free use prepared (precompiled?/fixed) packages.
See In graalpython what is difference between `ginstall` and `pip`?
Runnable code is at https://github.com/paulvi/graalpython-java-template