javapythonjavafxinputstreamcommand-pattern

How to read from InputStream that is constantly updating


I am developing an applicaton in Java that basically just collects input from user via GUI and feeds this data to a python script that does some file system operations. This python script is executed via Java Process class. Each script simply prints some data to its stdout, which I parse in the main java application when I call process.getInputStream().

I have one python script that basically does tail -f on a log file and prints it line by line. My java program parses this output using InputStream from that particular process. However, I feel like it's incredibly slow. I am appending this log to a JavaFX TextArea and it usually takes around 20s after the python script is executed that my java program starts picking up anything, but when I run the python script by itself, it runs as expected.

What I have tried so far: Parsing it using buffered reader like this:

InputStream is = this.currentTask.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(is, StandardCharsets.UTF_8));
String line;
while (this.currentTask.isAlive()){
   line = reader.readLine();
   if (line != null){
           final String finalLine = line;
           // JavaFX stuff so that the appending is on the main thread
           Platform.runLater(() -> output.appendText(finalLine + "\n"));
           continue;
   }
   // wait for the log to be updated 
   Thread.sleep(100);
}

When I step through the lines with a debugger, it gets stuck for a very long time in th line = reader.readLine(); part. Switching the reading method for another one provided by the BufferedReader(like read with a specified char buffer) does not help at all.

Any idea what might be the problem?

EDIT: added MCVE

Java program:

public static void main(String[] args) {
    Process process = null;
    try {
        process = Runtime.getRuntime().exec("python" + " " + "main.py");
    } catch (IOException e) {
        e.printStackTrace();
    }
    try {
        InputStream is = process.getInputStream();
        BufferedReader reader = new BufferedReader(new InputStreamReader(is, StandardCharsets.UTF_8));
        String line;
        while (process.isAlive()){
            line = reader.readLine();
            if (line != null){
                System.out.println(line);
                continue;
            }
            // wait for the log to be updated
            Thread.sleep(100);

        }
    } catch (IOException | InterruptedException e) {
        e.printStackTrace();
    }
}

Python program:

import time

i = 0
while True:
    print("Output ", i)
    i += 1
    time.sleep(0.5)

I noticed that the whole thing works as it should when I remove the sleep from python script. However, in the real app, the sleep is needed.


Solution

  • Ok I just solved it. Credit to this SO question.

    Java wasn't the problem, python simply wasn't flushing its output often enough and Java had nothing to read, that's all. I have to manualy flush my python print() functions and everything works as it should.