I have a CSV from a report generator that adds some non-CSV preamble, like:
Time Off Requests
My Org
Street Address
City, State ZIP
Col1,Col2,Col3
r1c1,r1c2,r1c3
r2c1,r2c2,r2c3
I need to discard the 6 lines before passing the file's io.Reader to csv.NewReader and trying a ReadAll(), so I need to ensure I don't read even a byte past the 6th line.
I originally thought of bufio.Scanner and calling Scan() in a loop 6 times, but then experimentally realized the "buf" in bufio means I cannot control where the buffered reads end and it will likely read past the true start of the CSV.
So I came up with just reading byte-by-byte till I've counted six line feeds (10):
f, _ := os.Open(csvPath)
// Read just past report-generated 6-line preamble
b := make([]byte, 1)
for i := 0; i < 6; {
f.Read(b)
if b[0] == 10 {
i++
}
}
r := csv.NewReader(f)
records, err = r.ReadAll()
...
And that works. But, is there a more idiomatic Go way?
You don't need to avoid using bufio
, in fact you should prefer using buffered IO when possible. What you can't do is use the original reader after accessing it through theĀ bufio.Reader
, i,e, don't pass the os.File
to csv.NewReader
after using bufio.NewReader
, continue using the bufio.Reader
which may contain data already read from the file.
Once you have a bufio.Reader
, you can make use of all the methods for reading parts of the stream, and don't need to worry about reading byte-by-byte.
buf := bufio.NewReader(f)
// The preamble is defined as 6 lines.
for i := 0; i < 6; i++ {
line, err := buf.ReadBytes('\n')
if err != nil {
log.Fatal(err)
}
fmt.Printf("line: %q\n", line)
}
r := csv.NewReader(buf)
records, err := r.ReadAll()
if err != nil {
log.Fatal(err)
}
fmt.Printf("records: %q\n", records)