I am running the following command to count the number of rows in a CSV file:
wc -l filename.csv
Sometimes the row count is correct and other times it is 1 less than the actual count. All of the files are in the same format with the same header row. My first suspicion is the header row but it doesn't seem like the command would differentiate between the header and other rows. The files are saved with the same encoding utf-8.
Is this an issue with the formatting of the CSV files and/or a nuance of the wc
command?
Assumptions:
\n
at the end of the last line.csv
data fields do not include embedded linefeeds (\n
); otherwise the count will return the number of lines in the file as opposed to the number of records in the fileSetup some test files:
$ printf "a\nb\nc" > file1 # no \n on the last line
$ printf "a\nb\nc\n" > file2
$ od -c file1
0000000 a \n b \n c
0000005 ^^-- terimination is missing
$ od -c file2
0000000 a \n b \n c \n
0000006 ^^-- terminate last line
We can see that wc -l
'misses' the last line from file1
:
$ wc -l file1
2 file1
$ wc -l file2
3 file2
One awk
approach that works for both files:
$ awk 'END {print NR}' file1
3
$ awk 'END {print NR}' file2
3
An alternative approach would have the OP go through and append a \n
to the last line of files where the \n
is missing.
A web search on bash insure last line has linefeed
would be a good place to start looking for solutions ...