linuxshellunixwhile-loopifs

Reading .txt file through shell script stops after 1st line


I want to create a shell that reads a text file.

The file, users_test.txt, looks like:

Headers - Group : Team : User

Group_A:Team_Red Team_Green:worker_1 worker_2 worker_2 worker_3
Group_B:Team_Blue Team_Black Team_Grey:worker_4 worker_2 worker_5 worker_6

I want to loop through the .txt file above which is colon separated. Here is my current solution.

#!/bin/bash
# set filename
FILENAME="users_test.txt"

# loop through file, each  line is divided into 3 fields separated by a colon (:) 
while IFS=':' read -r groups teams users
do 
    echo "group $groups has teams $teams and user $users"
    
    for team in $teams; do
        echo "hey: "$team
    done
    
    for group in $groups; do
        echo "hi: "$group
    done

    # Iterate the string variable using for loop
    for user in $users; do
        echo "hello: "$user
    done

done < "$FILENAME"

Why does it end after the first loop iteration?

output:

admin@computer:$ ./setup-test1.sh
group  has teams Team_Red Team_Green and user worker_1 worker_2 worker_2 worker_3
hey: Team_Red
hey: Team_Green
hi: Group_A
hello: worker_1
hello: worker_2
hello: worker_2
hello: worker_3
admin@computer5:$ 

It seems to work fine but it ends after the first line, as a result Group B isn't printed.

Can anyone tell my why the script only reads the first line containing Group_A but not the second line with Group_B?


Solution

  • Given your comment that your file doesn't end in a linefeed, 0a:

    $ printf 'foo\n' | xxd
    00000000: 666f 6f0a                                foo.
    
    $ printf 'foo' | xxd
    00000000: 666f 6f                                  foo
    

    here's the difference when trying to read such a file:

    $ while IFS= read -r line; do echo "line=$line"; done < <(printf 'foo\n')
    line=foo
    
    $ while IFS= read -r line; do echo "line=$line"; done < <(printf 'foo')
    $
    

    A file that doesn't have a linefeed at the end is not a valid text file and so trying to read it with any text processing tool (e.g. read - note "The standard input shall be a text file.") is undefined behavior. It's not worth wasting time on what any tool does with such a file and why, just don't create files like that if you want to do any further text processing on them. If your editor creates files without terminating linefeeds then get a new editor, e.g. vi.