csvgo

Idiomatic Go to precisely read N lines (and not a byte over) of text?


I have a CSV from a report generator that adds some non-CSV preamble, like:

Time Off Requests

My Org
Street Address
City, State  ZIP

Col1,Col2,Col3
r1c1,r1c2,r1c3
r2c1,r2c2,r2c3

I need to discard the 6 lines before passing the file's io.Reader to csv.NewReader and trying a ReadAll(), so I need to ensure I don't read even a byte past the 6th line.

I originally thought of bufio.Scanner and calling Scan() in a loop 6 times, but then experimentally realized the "buf" in bufio means I cannot control where the buffered reads end and it will likely read past the true start of the CSV.

So I came up with just reading byte-by-byte till I've counted six line feeds (10):

f, _ := os.Open(csvPath)

// Read just past report-generated 6-line preamble
b := make([]byte, 1)
for i := 0; i < 6; {
    f.Read(b)
    if b[0] == 10 {
        i++
    }
}

r := csv.NewReader(f)
records, err = r.ReadAll()
...

And that works. But, is there a more idiomatic Go way?


Solution

  • You don't need to avoid using bufio, in fact you should prefer using buffered IO when possible. What you can't do is use the original reader after accessing it through theĀ bufio.Reader, i,e, don't pass the os.File to csv.NewReader after using bufio.NewReader, continue using the bufio.Reader which may contain data already read from the file.

    Once you have a bufio.Reader, you can make use of all the methods for reading parts of the stream, and don't need to worry about reading byte-by-byte.

    buf := bufio.NewReader(f)
    // The preamble is defined as 6 lines.
    for i := 0; i < 6; i++ {
        line, err := buf.ReadBytes('\n')
        if err != nil {
            log.Fatal(err)
        }
        fmt.Printf("line: %q\n", line)
    }
    r := csv.NewReader(buf)
    records, err := r.ReadAll()
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("records: %q\n", records)
    

    Full Example