phpsearchweb-scrapingscraper

Data screaping based on Search engines


Is it possible to scrap the web based on Keywords using Search engines in PHP?

Like when some put keyword, the script will search google and render the results and then render the pages and scrap/extract the line that includes the matched keywords?

Any idea or library to refer to?


Solution

  • You can do that using google api https://developers.google.com/custom-search/json-api/v1/overview and a related php client https://github.com/google/google-api-php-client.

    Later on you need to write a web scraper to download the websites (curl) and parse the html parser (i.e. https://github.com/paquettg/php-html-parser).

    I would, however, not recommend php for the latter task. There are much more sophisticated scraping tools available for python (i.e. BeautifulSoup or Scrapy) that will make your life much MUCH easier than using php.