javaapache-beamdirect-runner

Missing options in DirectOptions class

The docs mention the following options: direct_num_workers and direct_running_mode as well as setting the streaming option.

All of these are missing from the DirectOptions class

Also when trying to set those from args the following exception is thrown:

java.lang.IllegalArgumentException: Class interface org.apache.beam.runners.direct.DirectOptions missing a property named 'direct_running_mode'.
        at org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1613)
        at org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:117)
        at org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:300)

Has someone managed to use these? How?

Solution

  • These options are relevant for the Python SDK only, it’s actually not clear as these options are documented in the caveats section for both sdks, it should be hidden when documenting the Java SDK, feel free to report the documentation issue.

    These options are part of the Python FnApiRunner.