pythonweb-scrapingbeautifulsoup

Screen Scraping a Javascript based webpage in Python

I am working on a screen scraping tool in Python. But, as I look through the source of the webpage, I noticed that most of the data is coming through Javascript.

Any idea, how to scrape javascript based webpage ? Any tool in Python ?

Thanks

Solution

  • Scraping javascript-based webpages is possible with selenium. In particular, try the Selenium WebDriver.