bashloopslynx

Lynx is stopping loop?


I'll just apologize beforehand; this is my first ever post, so I'm sorry if I'm not specific enough, if the question has already been answered and I just didn't look hard enough, and if I use incorrect formatting of some kind.

That said, here is my issue: In bash, I am trying to create a script that will read a file that lists several dozen URL's. Once it reads each line, I need it to run a set of actions on that, the first being to use lynx to navigate to the website. However, in practice, it will run once perfectly on the first line. Lynx goes, the download works, and then the subsequent renaming and organizing of that file go through as well. But then it skips all the other lines and acts like it has finished the whole file.

I have tested to see if it was lynx causing the issue by eliminating all the other parts of the code, and then by just eliminating lynx. It works without Lynx, but, of course, I need lynx for the rest of the output to be of any use to me. Let me just post the code:

!#/bin/bash

while read line; do
echo $line
  lynx -accept_all_cookies  $line
echo "lynx done"
  od -N 2 -h *.zip | grep "4b50"
echo "od done, if 1 starting..."
    if [[ $? -eq 0 ]]
    then ls *.*>>logs/zips.log
    else
    od -N 2 -h *.exe | grep "5a4d"
echo "if 2 starting..."     
      if [[ $? -eq 0 ]]
      then ls *.*>>logs/exes.log
      else
      od -N 2 -h *.exe | grep "5a4d, 4b50"
echo "if 3 starting..."
        if [[ $? -eq 1 ]]
        then
        ls *.*>>logs/failed.log
        fi
echo "if 3 done"
      fi
echo "if 2 done"
    fi
echo "if 1 done..."
  FILE=`(ls -tr *.* | head -1)`
  NOW=$(date +"%m_%d_%Y")
echo "vars set"
  mv $FILE "criticalfreepri/${FILE%%.*}(ZCH,$NOW).${FILE#*.}" -u
echo "file moved"
  rm *.zip *.exe
echo "file removed"
done < "lynx"

$SHELL

Just to be sure, I do have a file called "lynx" that contains the urls separated by a return each. Also, I used all those "echo"s to do my own sort of debugging, but I have tried it with and without the echo's. When I execute the script, the echo's all show up...

Any help is appreciated, and thank you all so much! Hope I didn't break any rules on this post!

PS: I'm on Linux Mint running things through the "terminal" program. I'm scripting with bash in Gedit, if any of that info is relevant. Thanks!

EDIT: Actually, the echo tests repeat for all three lines. So it would appear that lynx simply can't start again in the same loop?

Here is a simplified version of the script, as requested:

!#/bin/bash

while read -r line; do
echo $line
lynx $line
echo "lynx done"
done < "ref/url" 
read "lynx"

$SHELL

Note that I have changed the sites the "url" file goes to:

`www.google.com

www.majorgeeks.com

http://www.sophos.com/en-us/products/free-tools/virus-removal-tool.aspx`


Solution

  • I should have answered this question a long time ago. I got the program working, it's now on Github!

    Anyway, I simply had to wrap the loop inside a function. Something like this:

    progdownload () {
      printlog "attmpting download from ${URL}"
      if echo "${URL}" | grep -q "http://www.majorgeeks.com/" ; then
    lynx -cmd_script="${WORKINGDIR}/support/mgcmd.txt" --accept-all-cookies ${URL}
      else wget ${URL}
      fi
      }
    URL="something.com"
    progdownload