Jlink - including a directory that contains a custom python script in a JavaFX application

I need to include a directory containing a python script and binaries that need to be executed by the script based on the parsed arguments in the JavaFX application.

The project is modular and built using Maven (although the modular part is not such an important piece of information).

When built using the maven run configuration, application works properly but for the purpose of creating a runtime image I stumble upon the problem of not having the script executed when I run the generated launcher .bat script in the "bin" folder of the "target".

For the purpose of generating the runtime, I have put the script directory in the project "resources" folder. The script is executed from the Java code using the Java Runtime.

Let's say the code looks like this:

pyPath = Paths.get("src/main/resources/script/main.py").toAbsolutePath().toString();
command = "python"+pyPath+args;
runtime = Runtime.getRuntime();
process = runtime.exec(command);

And pom.xml file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>gui</artifactId>
  <version>1.0-SNAPSHOT</version>
  <name>gui</name>
  <packaging>jar</packaging>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <junit.version>5.8.2</junit.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.openjfx</groupId>
      <artifactId>javafx-controls</artifactId>
      <version>18</version>
    </dependency>
    <dependency>
      <groupId>org.openjfx</groupId>
      <artifactId>javafx-fxml</artifactId>
      <version>18</version>
    </dependency>
    <dependency>
      <groupId>org.controlsfx</groupId>
      <artifactId>controlsfx</artifactId>
      <version>11.1.1</version>
    </dependency>
    <dependency>
      <groupId>com.dlsc.formsfx</groupId>
      <artifactId>formsfx-core</artifactId>
      <version>11.3.2</version>
      <exclusions>
        <exclusion>
          <groupId>org.openjfx</groupId>
          <artifactId>*</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.kordamp.ikonli</groupId>
      <artifactId>ikonli-javafx</artifactId>
      <version>12.3.0</version>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-api</artifactId>
      <version>${junit.version}</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <version>${junit.version}</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>com.jfoenix</groupId>
      <artifactId>jfoenix</artifactId>
      <version>9.0.10</version>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.panteleyev</groupId>
        <artifactId>jpackage-maven-plugin</artifactId>
        <version>1.5.2</version>
        <configuration>
          <name>gui</name>
          <appVersion>1.0.0</appVersion>
          <vendor>1234</vendor>
          <destination>target/dist</destination>
          <module>com.example.gui/com.example.gui.Application</module>
          <runtimeImage>target/example-gui</runtimeImage>
          <winDirChooser>true</winDirChooser>
          <winPerUserInstall>true</winPerUserInstall>
          <winShortcut>true</winShortcut>
          <winMenuGroup>Applications</winMenuGroup>
          <icon>${project.basedir}/main/resources/img/icon.ico</icon>
          <javaOptions>
            <option>-Dfile.encoding=UTF-8</option>
          </javaOptions>
        </configuration>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.10.1</version>
        <configuration>
          <source>18</source>
          <target>18</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.openjfx</groupId>
        <artifactId>javafx-maven-plugin</artifactId>
        <version>0.0.8</version>
        <executions>
          <execution>
            <id>default-cli</id>
            <configuration>
              <mainClass>com.example.gui/com.example.gui.Application</mainClass>
              <launcher>gui-launcher</launcher>
              <jlinkZipName>gui</jlinkZipName>
              <jlinkImageName>gui</jlinkImageName>
              <jlinkVerbose>true</jlinkVerbose>
              <noManPages>true</noManPages>
              <stripDebug>true</stripDebug>
              <noHeaderFiles>true</noHeaderFiles>
              <options>
                <option>--add-opens</option><option>javafx.graphics/com.sun.javafx.scene=ALL-UNNAMED</option>
                <option>--add-opens</option><option>javafx.controls/com.sun.javafx.scene.control.behavior=ALL-UNNAMED</option>
                <option>--add-opens</option><option>javafx.controls/com.sun.javafx.scene.control=ALL-UNNAMED</option>
                <option>--add-opens</option><option>javafx.base/com.sun.javafx.binding=ALL-UNNAMED</option>
                <option>--add-opens</option><option>javafx.graphics/com.sun.javafx.stage=ALL-UNNAMED</option>
                <option>--add-opens</option><option>javafx.base/com.sun.javafx.event=ALL-UNNAMED</option>
                <option>--add-exports</option><option>javafx.controls/com.sun.javafx.scene.control.behavior=ALL-UNNAMED</option>
                <option>--add-exports</option><option>javafx.controls/com.sun.javafx.scene.control=ALL-UNNAMED</option>
                <option>--add-exports</option><option>javafx.base/com.sun.javafx.binding=ALL-UNNAMED</option>
                <option>--add-exports</option><option>javafx.graphics/com.sun.javafx.stage=ALL-UNNAMED</option>
                <option>--add-exports</option><option>javafx.graphics/com.sun.javafx.scene=ALL-UNNAMED</option>
                <option>--add-exports</option><option>javafx.base/com.sun.javafx.event=ALL-UNNAMED</option>
              </options>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

*Note: additional options for the javafx-maven-plugin are added for the jfoenix package compatibility

Also module-info.java

module com.example.gui {
    requires javafx.controls;
    requires javafx.fxml;

    requires org.controlsfx.controls;
    requires com.dlsc.formsfx;
    requires org.kordamp.ikonli.javafx;
    requires com.jfoenix;

    opens com.example.gui to javafx.fxml;
    exports com.example.gui;
}

Now the question is how do I include the script in the application runtime image, have it executed when I call the generated .bat for the application and finally packed using the jpackage?

Solution

Problems

The src/main/resources directory only exists in your project sources. It does not exist in the build output, and it definitely does not exist in your deployment location. In other words, using:

var pyPath = Paths.get("src/main/resources/script/main.py").toAbsolutePath().toString();

Will only work when your working directory is your project directory. It's also reading the "wrong" main.py resource, as the "correct" one will be in your target directory. Additionally, resources are not files. You must access resources using the resource-lookup API. For example:

var pyPath = getClass().getResource("/script/main.py").toString();

And note src/main/resources is not included in the resource name.

Executing the Script

But even after you correctly access the resource you still have a problem. Your script is a resource, which means it will be embedded in a JAR file or custom run-time image when deployed. I strongly doubt Python will know how to read and execute such a resource. This means you need to find a way to make the Python script a regular file on the host computer.

Potential Solutions

I can think of at least three approaches that may solve the problems described above. As I only have Windows, I can't promise the below examples will work on other platforms, or otherwise be easily translated for other platforms.

My examples don't include JavaFX, as I don't believe that's necessary to demonstrate how to include a Python script that is executed at runtime.

Here are some common aspects between all solutions.

module-info.java:

module com.example {}

main.py:

print("Hello from Python!")

1. Extract the Script at Runtime

One approach is to extract the Python script to a known location on the host computer at runtime. This is likely the most versatile solution, as it doesn't depend much on the way you deploy your application (jar, jlink, jpackage, etc.).

This example extracts the script to a temporary file, but you can use another location, such as an application-specific directory under the user's home directory. You can also code it to extract only if not already extracted, or only once per application instance.

I think this is the solution I would use, at least to start with.

Project structure:

|   pom.xml
|
\---src
    \---main
        +---java
        |   |   module-info.java
        |   |
        |   \---com
        |       \---example
        |           \---app
        |                   Main.java
        |
        \---resources
            \---scripts
                    main.py

Main.java:

package com.example.app;

import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;

public class Main {

  public static void main(String[] args) throws Exception {
    Path target = Files.createTempDirectory("sample-1.0").resolve("main.py");
    try {
      // extract resource to temp file
      try (InputStream in = Main.class.getResourceAsStream("/scripts/main.py")) {
        Files.copy(in, target);
      }

      String executable = "python";
      String script = target.toString();

      System.out.printf("COMMAND: %s %s%n", executable, script); // log command
      new ProcessBuilder(executable, script).inheritIO().start().waitFor();
    } finally {
      // cleanup for demo
      Files.deleteIfExists(target);
      Files.deleteIfExists(target.getParent());
    }
  }
}

pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>sample</artifactId>
  <version>1.0</version>

  <name>sample</name>
  <packaging>jar</packaging>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.release>18</maven.compiler.release>
  </properties>

  <build>
    <plugins>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.10.1</version>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <version>3.2.2</version>
        <configuration>
          <archive>
            <manifest>
              <mainClass>com.example.app.Main</mainClass>
            </manifest>
          </archive>
        </configuration>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jlink-plugin</artifactId>
        <version>3.1.0</version>
        <executions>
          <execution>
            <id>default-cli</id>
            <phase>package</phase>
            <goals>
              <goal>jlink</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <classifier>win</classifier>
        </configuration>
      </plugin>

      <plugin>
        <groupId>org.panteleyev</groupId>
        <artifactId>jpackage-maven-plugin</artifactId>
        <version>1.5.2</version>
        <executions>
          <execution>
            <id>default-cli</id>
            <phase>package</phase>
            <goals>
              <goal>jpackage</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <type>APP_IMAGE</type>
          <runtimeImage>${project.build.directory}/maven-jlink/classifiers/win</runtimeImage>
          <module>com.example/com.example.app.Main</module>
          <destination>${project.build.directory}/image</destination>
          <winConsole>true</winConsole>
        </configuration>
      </plugin>

    </plugins>
  </build>

</project>

Build the project with:

mvn package

And then execute the built application image:

> .\target\image\sample\sample.exe

COMMAND: python C:\Users\***\AppData\Local\Temp\sample-1.015076039373849618085\main.py
Hello from Python!

2. Place the Script in the "Application Directory"

Disclaimer: I don't know if doing this is a smart or even supported approach.

This solution makes use of --input to place the script in the "application directory" of the application image. You can then get a reference to this directory by setting a system property via --java-options and $APPDIR. Note I tried to get this to work with the class-path, so as to not need the $APPDIR system property, but everything I tried resulted in Class#getResource(String) returning null.

The application directory is the app directory shown in this documentation.

As this solution places the Python script with the rest of the application image, which means it's placed in the installation location, you may be more likely to run into file permission issues.

Given the way I coded Main.java, this demo must be executed only after packaging with jpackage. I suspect there's a more robust way to implement this solution.

Project structure:

|   pom.xml
|
+---lib
|   \---scripts
|           main.py
|
\---src
    \---main
        \---java
            |   module-info.java
            |
            \---com
                \---example
                    \---app
                            Main.java

Main.java:

package com.example.app;

import java.nio.file.Path;

public class Main {

  public static void main(String[] args) throws Exception {
    String executable = "python";
    String script = Path.of(System.getProperty("app.dir"), "scripts", "main.py").toString();

    System.out.printf("COMMAND: %s %s%n", executable, script); // log command
    new ProcessBuilder(executable, script).inheritIO().start().waitFor();
  }
}

pom.xml:

(this is only the <configuration> of the org.panteleyev:jpackage-maven-plugin plugin, as everything else in the POM is unchanged from the first solution)

<configuration>
  <type>APP_IMAGE</type>
  <runtimeImage>${project.build.directory}/maven-jlink/classifiers/win</runtimeImage>
  <input>lib</input>
  <javaOptions>
    <javaOption>-Dapp.dir=$APPDIR</javaOption>
  </javaOptions>
  <module>com.example/com.example.app.Main</module>
  <destination>${project.build.directory}/image</destination>
  <winConsole>true</winConsole>
</configuration>

Build the project with:

mvn package

And then execute the built application image:

> .\target\image\sample\sample.exe

COMMAND: python C:\Users\***\Desktop\sample\target\image\sample\app\scripts\main.py
Hello from Python!

3. Add the Script as Additional "App Content"

Disclaimer: Same as the disclaimer for the second solution.

This would make use of the --app-content argument when invoking jpackage. Unfortunately, I could not figure out how to configure this using Maven, at least not with the org.panteleyev:jpackage-maven-plugin plugin. But essentially this solution would have been the same as the second solution above, but with --input removed and --app-content lib/scripts added. And a slight change to how the script Path is resolved in code.

The --app-content argument seems to put whatever directories/files are specified in the root of the generated application image. I'm not sure of a nice convenient way to get this directory, as the application image structure is slightly different depending on the platform. And as far as I can tell, there's no equivalent $APPDIR for the image's root directory.