gocronscheduled-tasksscheduler

How to Prevent Pilled Up Job Executions in GoCron if Job Takes Longer than Scheduled Interval?


I'm using the github.com/go-co-op/gocron package in my Go project to schedule tasks. The job is set to run every 2 minutes, but there's a scenario where the task might take longer than the interval (e.g., the job takes 7 minutes to complete). After the long-running job completes, multiple job instances get triggered back-to-back, which causes overlapping executions.

func newScheduler(loc *time.Location) {
    e := gocron.NewScheduler(loc)
    e.SingletonModeAll()

    e.Every("2m").Do(func() {
        Logger.Warn("Task Started at: " + time.Now().Format("2006-01-02 15:04:05"))
        err := tasks.MyTask()
        if err != nil {
            Logger.Error("Error occurred during task execution: ", err)
        }
        Logger.Warn("Task Completed at: " + time.Now().Format("2006-01-02 15:04:05"))
    })

    e.StartAsync()
    select {}
}

Problem: When the job takes more time than the scheduled interval (e.g., 7 minutes instead of 2), multiple instances of the job get executed back-to-back after the long-running job finishes. This causes performance issues and overlapping jobs.

Question: Is there a way to ensure that only one instance of the job runs after execution of the job, and subsequent jobs wait until the previous instance finishes before starting and should only once based on the next interval?

What I've tried: I've enabled SingletonModeAll(), but it doesn't prevent the execution of the back-to-back jobs when the interval is missed due to a long-running task.

Desired Outcome: I want to ensure that after execution of the job, it should ignore the pilled-up job and should not run back to back, even if the job takes longer than the scheduled interval.

Any guidance on how to handle this in GoCron or alternative approaches would be greatly appreciated!


Solution

  • You are using the old v1 API, which says:

    SingletonModeAll prevents new jobs from starting if the prior instance of the particular job has not yet completed its run

    Warning: do not use this mode if your jobs will continue to stack up beyond the ability of the limit workers to keep up.

    What you want to use is SetMaxConcurrentJobs:

    limits how many jobs can be running at the same time.

    With RescheduleMode:

    the default is that if a limit on maximum concurrent jobs is set and the limit is reached, a job will skip it's run and try again on the next occurrence in the schedule

    so

        e.SetMaxConcurrentJobs(1, gocron.RescheduleMode)
    

    Note that I would suggest switching to github.com/go-co-op/gocron/v2 which has WithLimitConcurrentJobs instead (Example):

        s, err := gocron.NewScheduler(gocron.WithLimitConcurrentJobs(1, gocron.LimitModeReschedule))