windowsbatch-filepaginationxidel

xidel how to follow pagination html and extract URL?

On windows 7 in batch and xidel I test on a website with pagination like this example :

link1

link2

link3

1 2 3 4 5 6 7 8 9 10 Next

i find a way to have first 10 links :

xidel.exe https://www.website.es/search?q=xidel+follow+pagination^&start=0 --extract "//a/extract(@href, 'url[?]q=([^&]+)&', 1)[. != '']"

but when i try to follow to page 2 or page (n) with

-f "<A class="fl">{.}</A>"

or

--follow "//a/[@class='nav']"

nothink work, can you give me some help or some example ?

Thanks.

Solution

  • Reino is right. But querying Google can also be done like this:

    xidel -s "https://www.google.com" ^
      -f "form(//form,{'q':'xidel follow pagination','num':'25'})" ^
      -e "//a/extract(@href,'url\?q=(.+?)&',1)[.]"