[SOLVED] Process to pre-validate Azkaban flows

Process to pre-validate Azkaban flows

I would like to validate my Azkaban flows before I upload them to the server, as simple as that. Do we have a plugin or something to do that? If not what are the classes within Azkaban github that do this validation? I could just adapt them and use it to do the validation.

Solution

I recommend using the Azkaban "CLI" which you can find here: https://github.com/mtth/azkaban

You submit your project from the terminal and get feedback like so:

$ azkaban build -c --project my_project
  > Validator Directory Flow reports errors:<ul><li>my_flow_1 cannot 
    find dependency my_other_flow</li></ul>

$ azkaban build -c --project my_project
  > Project my_project successfully built and uploaded (id: 20, size: 28.2kB, upload: 2).
    Details at https://192.168.0.1:8443/manager?project=my_project

I have a fleshed out example of the job.py format for the CLI at: https://github.com/joeharris76/azkaban_examples