gogo-context

Pattern for cancelling a group of long-running goroutines


I would like to execute a set of long-running tasks in a loop and cancel all of them if any one fails. To do that I wrap each task in a Manager task that spawns off the actual long-running task and then goes back to waiting for the context to be cancelled due to an error in any other task in the loop.

My question is: "Does cancelling the individual Manager tasks also terminate the spawned off long-running task, given that the actual task has no reference to the cancellable context in the parent?"

The "working" code below should make my question clear. I understand that all instances of readWithCancel will return, but will each corresponding readCSV child terminate as a result.

Thanks as always

func main() {
    files := []string{"../data/file1.csv", "../data/file2.csv", "../data/file3.csv"}

    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()

    var wg sync.WaitGroup
    for _, file := range files {
        wg.Add(1)
        fileLocal := file
        go func() {
            defer wg.Done()
            if err := readWithCancel(ctx, fileLocal); err != nil {
                fmt.Println(fileLocal, err)
                cancel()
            }
        }()
    }
    wg.Wait()
}

func readWithCancel(ctx context.Context, file string) error {

    ch := make(chan error)

    go func() {
        ch <- readCSV(file)
    }()

    select {
    case err := <-ch:
        return err
    case <-ctx.Done():
        return ctx.Err()
    }
}

func readCSV(file string) error {

    f, err := os.Open(file)
    if err != nil {
        return fmt.Errorf("opening file %w", err)
    }

    cr := csv.NewReader(f)
    linenum := 0
    for {
        record, err := cr.Read()
        if err != nil {
            if errors.Is(err, io.EOF) {
                return nil
            }
            return err
        }

        fmt.Printf("%s: %s\n", file, record)
        linenum++
    }
}

Solution

  • The short answer is "no." Goroutines have no inherent relationship to each other (parent-child, etc.). A goroutine runs until its top-level function (the function specified in the go statement) terminates or the program terminates.

    In the example code you provided, an error will result in the program terminating since canceling the context will cause the goroutines you're waiting for to terminate, causing wg.Wait to return. When that happens, the goroutines running readCSV will terminate with the program, but I don't think that's the solution you're looking for.