pythonweb-scrapinglxmllxml.html

Web scraping data from an interactive chart that changes with cursor position on the screen

I am trying to web scrape data from this url, http://poker.srv.ualberta.ca/preflop. On the page you see an interactive table who only shows the preflop actions if you move your cursor over the different squares. I checked the source code and all the info is in the class stat summary, but stat summary changes with cursor position. Is there a way to use lxml and requests to extrapolate the ENTIRE table? I am able to do it for a single position but not all.

My code so far:

from lxml import html
import requests
page = requests.get("http://poker.srv.ualberta.ca/preflop")
tree = html.fromstring(page.content)
Solution

  • I have seen that there is no AJAX call to load that data behind graphs.

    Only way they can load that graph's data is from those JS files on their site.

    I suggest your to look over all the JS files and see how they are computing that values.

    Here are some of the JS files on their servers.

    http://poker.srv.ualberta.ca/data/crPreflop.js

    and

    http://poker.srv.ualberta.ca/data/cPreflop.js

    You will have to spend a day studying data inside those files and you will figure out.