javadockerdocker-composeargumentsapache-flink

How do i pass Program Arguments From Apache Flink 2.0.0 Web Gui to my Job properly?


I need to submit a new Apache Flink Job from the Web GUI instanced in a docker container in Session Mode and i need to pass some arguments to the main function of my job written in Java.

I'm trying to pass these arguments using the Program Arguments field located in the Submit New Job section as shown in the image below

enter image description here

The thing is, when i try to read these arguments from my main function the args array is always empty and i can't parse them properly.

Here is how i parsed the arguments in Flink 2.0.0.

public static void main(String[] args) throws Exception {
    loggingObj.info("ARGS: {}", Arrays.toString(args))
  
    // GET PARAMETERS FROM FLINK GUI
    ParameterTool parameters = ParameterTool.fromArgs(args);

    double windowDurationS = parameters.getDouble("windowDurationS", 5.0);
    double watermarkIdlenessS = parameters.getDouble("watermarkIdlenessS",4.0);
    double queryIntervalS = parameters.getDouble("queryIntervalS", 4.0);
    String processingFunction = parameters.get("processingFunction", "All Supported Functions").trim();

    loggingObj.info("PARAMETERS: {}", parameters.toMap());
    loggingObj.info("PROCESSING FUNCTION: {}", processingFunction);

I also tried to change the format in which I pass the arguments.

I tried

-param1 value1
--param1 value1
param1="value1"
param1=value1

but none of them seems to work. When i log the arguments from my code the list is always empty as shown in the logs

INFO  demoFlink.DataStreamJob        [] - ARGS: []
INFO  demoFlink.DataStreamJob        [] - PARAMETERS: {}
INFO  demoFlink.DataStreamJob.       [] - PROCESSING FUNCTION: All Supported Functions

I also checked the entry point of my job and everything is correct, the is just one main and all others log seems to work just fine. Also, if I try to run the same job with the arguments that i need locally everything seems to be okay.

Do i need to do some further configurations in the docker-compose.yaml file or there is something that I'm missing?


Solution

  • Had same issue, figured it out. Looks like a bug, UI is sending programArgs argument in the request JSON body, it expects programArgsList instead. Example curl which works for me:

    curl -X POST -H "Content-Type: application/json" --data '{"entryClass":"org.example.SomeClass","parallelism":null,"programArgsList":["--something", "pretty"],"savepointPath":null,"allowNonRestoredState":null}' http://<ip-of-my-flink-ui>:8081/jars/<id_of_my_flink_job>_<name-of-my-jar>.jar/run