pythonluigi

How to schedule workflow in Luigi?


I am able to instantaneously execute a pipeline/workflow in luigi using the following:

luigi --module mypipeline mypipeline --local-scheduler

But how can I add a schedule to it, for example executing it every 10 days?

I wasn't able to find scheduler examples or applicable sample code in the documentation


Solution

  • Luigi does not include its own triggering, so you have to rely on an external scheduler such as crontab to actually trigger the workflows.

    For details, please refer to this section of the docs