Go func for running external command, occasionally getting error: file already closed

My function

func RunCommand(cmdPath string, cmdArgs string) (error) {
    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    cmd := exec.CommandContext(ctx, cmdPath, cmdArgs)
    stdout, err := cmd.StdoutPipe()
    scanner := bufio.NewScanner(stdout)
    err := cmd.Start()
    var wg sync.WaitGroup
    wg.Add(1)

    go func() {
        defer wg.Done()
        for scanner.Scan() {
            doSomething(scanner.Text())
        }
        if scanner.Err() != nil {
            log.Errf("Error reading stdout, %v", scanner.Err())
        }
    }()

    done := make(chan error, 1)
    go func() {
        done <- cmd.Wait()
    }()
    wg.Wait()

    return nil
}

I am only able to reproduce this issue a few times where i get this error: Error reading stdout, read |0: file already closed. My understanding is i am closing stdout prior to checking scanner.Err() but why is that happening. I am using cmd.Wait() in a go routine which should wait for scanner.Err() and then close stdout right?

The error is likely to be caused by the subprocess terminating unexpectedly.

Consider this example. We start a bash loop printing every quarter second. But one second later, a goroutine kills the process by pid. Its output follows.

package main

import (
    "bufio"
    "context"
    "log"
    "os"
    "os/exec"
    "sync"
    "syscall"
    "time"
)

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    cmd := exec.CommandContext(ctx, "sh", "-c", "for i in $(seq 1 10); do echo $i; sleep 0.25; done")

    cmd.Stderr = os.Stderr
    stdout, err := cmd.StdoutPipe()
    if err != nil {
        log.Fatal(err)
    }
    defer stdout.Close()
    scanner := bufio.NewScanner(stdout)
    if err := cmd.Start(); err != nil {
        log.Fatal(err)
    }
    go func() { time.Sleep(1 * time.Second); syscall.Kill(cmd.Process.Pid, syscall.SIGKILL) }()
    var wg sync.WaitGroup
    wg.Add(1)

    go func() {
        defer wg.Done()
        for scanner.Scan() {
            log.Print(scanner.Text())
        }
        if scanner.Err() != nil {
            log.Printf("Error reading stdout, %v", scanner.Err())
        }
    }()

    done := make(chan error, 1)
    go func() {
        done <- cmd.Wait()
    }()
    select {
    case <-ctx.Done():
        log.Fatal(ctx.Err())
    case err := <-done:
        if err != nil {
            log.Fatal(err)
        }
    }
    wg.Wait()

}

2024/09/18 22:34:44 1
2024/09/18 22:34:45 2
2024/09/18 22:34:45 3
2024/09/18 22:34:45 4
2024/09/18 22:34:45 Error reading stdout, read |0: file already closed
2024/09/18 22:34:45 signal: killed
exit status 1

If you want to handle the error differently, you can compare it to os.ErrClosed:

  if err := scanner.Err(); err != nil {
      if errors.Is(err, os.ErrClosed) {
        log.Printf("Program ended unexpectedly")
      } else {
        log.Printf("Error reading stdout: %s", err.Error())
      }
    }