phpmysqldatabase

mysql database comparison before insert

So I am scraping a site for urls and storing these urls in a db I am looking for the lightest way to compare the new scraped urls with the urls stored to only add the ones that arent currently stored in the db. I am working in php with a mysql db.

Solution

  • Give the URL a UNIQUE index. In your query :

    INSERT IGNORE INTO table (field1,field2) VALUES ('value1','value2')

    This will skip the url when they already exist in the database.