bashprintfubuntu-14.04ubuntu-16.04printf-debugging

Ubuntu (14 & 16) Bash errors with printf loops from input containing lowercase "n" characters


I have some bash scripts I have been running on Ubuntu 14.04 and 16.04 for well over a year now. Some recent Ubuntu update has broken bash and I cannot figure out how to sort this out.

Example:

#!/bin/bash
INPUT=myinput.txt
OUTPUT=myoutput.txt
ACTION1="0;"

cat $INPUT | while read LINE
do
printf "\t\"~${LINE}\"\t\t$ACTION1\n" >> $OUTPUT
done

my script then loops through the file doing the printf statement as follows but this is the output produced

"~jetmo" 0;
"~spamme" 0;
"~baidu" 0;

example contents of myinput.txt

jetmon
spammen
baidu

Input lines containing a lowercase n character gets stripped out ?? Am I missing some major change in bash that occured, this started happening a week ago and is driving me insane now.

I have tried changing #!/bin/bash to #!/bin/sh with same results.

Also tried this method of looping through the input file and still get the same results.

#!/bin/bash
INPUT=myinput.txt
OUTPUT=myoutput.txt
ACTION1="0;"
while read LINE
do
printf "\t\"~${LINE}\"\t\t$ACTION1\n" >> $OUTPUT
done < $INPUT

Tried suggestion from comments using echo

echo -e -n "\t\"~${LINE}\"\t\t$ACTION1\n" >> $OUTPUT

and now I get this output

-e -n "~jetmo" 0;
-e -n "~spamme" 0;
-e -n "~baidu" 0;

Running hexdump -C myinput.txt gives this output

00000000  6a 65 74 6d 6f 6e 0a 73  70 61 6d 6d 65 6e 0a 62  |jetmon.spammen.b|
00000010  61 69 64 75 0a 0a                                 |aidu..|
00000016

Also changed all variable names to lowercase as suggested by Michael but still getting the same results.

#!/bin/bash
input=myinput.txt
output=myoutput.txt
action1="0;"

cat $input | while read line
do
printf "\t\"~${line}\"\t\t$action1\n" >> $output
done

THE SOLUTION The Mystery is Solved

thanks to everyone, but https://stackoverflow.com/users/96588/l0b0 nailed it on the head. I had an IFS=$'\n' hiding earlier on in my script.

I now have this final, better and safer formatting of printf thanks to the recommendations of Nahuel and Charles and it is working 100% ... cannot thank you all enough.

#!/bin/bash
input=myinput.txt
output=myoutput.txt
action1="0;"
while IFS= read -r LINE
do
printf '\t"~%s"\t\t%s\n' "${LINE}" "$ACTION1" >> "$output"
done < $input1

Solution

  • This can happen if the internal field separator ($IFS) contains the letter n:

    $ IFS=$' \t\nn' read line <<< foon
    $ printf '%q\n' "$line"
    foo
    

    This is a fairly common mistake. Here is the correct value:

    $ printf '%q\n' "$IFS"
    $' \t\n'
    

    Example:

    $ IFS=$' \t\n' read line <<< foon
    $ printf '%q\n' "$line"
    foon