I have some bash scripts I have been running on Ubuntu 14.04 and 16.04 for well over a year now. Some recent Ubuntu update has broken bash and I cannot figure out how to sort this out.
Example:
#!/bin/bash
INPUT=myinput.txt
OUTPUT=myoutput.txt
ACTION1="0;"
cat $INPUT | while read LINE
do
printf "\t\"~${LINE}\"\t\t$ACTION1\n" >> $OUTPUT
done
my script then loops through the file doing the printf statement as follows but this is the output produced
"~jetmo" 0;
"~spamme" 0;
"~baidu" 0;
example contents of myinput.txt
jetmon
spammen
baidu
Input lines containing a lowercase n character gets stripped out ?? Am I missing some major change in bash that occured, this started happening a week ago and is driving me insane now.
I have tried changing #!/bin/bash to #!/bin/sh with same results.
Also tried this method of looping through the input file and still get the same results.
#!/bin/bash
INPUT=myinput.txt
OUTPUT=myoutput.txt
ACTION1="0;"
while read LINE
do
printf "\t\"~${LINE}\"\t\t$ACTION1\n" >> $OUTPUT
done < $INPUT
Tried suggestion from comments using echo
echo -e -n "\t\"~${LINE}\"\t\t$ACTION1\n" >> $OUTPUT
and now I get this output
-e -n "~jetmo" 0;
-e -n "~spamme" 0;
-e -n "~baidu" 0;
Running hexdump -C myinput.txt gives this output
00000000 6a 65 74 6d 6f 6e 0a 73 70 61 6d 6d 65 6e 0a 62 |jetmon.spammen.b|
00000010 61 69 64 75 0a 0a |aidu..|
00000016
Also changed all variable names to lowercase as suggested by Michael but still getting the same results.
#!/bin/bash
input=myinput.txt
output=myoutput.txt
action1="0;"
cat $input | while read line
do
printf "\t\"~${line}\"\t\t$action1\n" >> $output
done
THE SOLUTION The Mystery is Solved
thanks to everyone, but https://stackoverflow.com/users/96588/l0b0 nailed it on the head. I had an IFS=$'\n'
hiding earlier on in my script.
I now have this final, better and safer formatting of printf thanks to the recommendations of Nahuel and Charles and it is working 100% ... cannot thank you all enough.
#!/bin/bash
input=myinput.txt
output=myoutput.txt
action1="0;"
while IFS= read -r LINE
do
printf '\t"~%s"\t\t%s\n' "${LINE}" "$ACTION1" >> "$output"
done < $input1
This can happen if the internal field separator ($IFS
) contains the letter n
:
$ IFS=$' \t\nn' read line <<< foon
$ printf '%q\n' "$line"
foo
This is a fairly common mistake. Here is the correct value:
$ printf '%q\n' "$IFS"
$' \t\n'
Example:
$ IFS=$' \t\n' read line <<< foon
$ printf '%q\n' "$line"
foon