phpcurlweb-scrapinginspect-elementview-source

page source code is showing different code and inspect element showing different code


while scraping a site when I view code with inspect element code is different and when I view the source code of a site then code is different.
What I want using curl in PHP how can i get the code which I can see using inspect element.
Page getting details through ajax call but there is not href in that ajax call
href which I need to get details of particular page.

Here is the curl request

    $login = curl_init();
    curl_setopt($login, CURLOPT_URL,www.example.com);
    curl_setopt($login, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:42.0) Gecko/20100101 Firefox/42.0');
    curl_setopt($login, CURLOPT_RETURNTRANSFER, TRUE);
    curl_setopt($login, CURLOPT_FOLLOWLOCATION, TRUE);
    curl_setopt($login, CURLOPT_SSL_VERIFYPEER, FALSE);
    curl_setopt($login, CURLOPT_TIMEOUT, 40000);
    $html = curl_exec ($login);

Solution

  • Well you can use the Phantom JS. It is a scriptable headless web browser. It is based on web kit and it supports many web standards. It can be used for taking screenshots, inspecting page elements. It can be called from Php so you should be able inspect the page elements from Php in the same way as you inspect the page elements from a browsers developer toolbox. This link shows how to use the Phantom JS from Php: https://github.com/jonnnnyw/php-phantomjs