When I run the code after a while it gives me the code: 429 which means that too many requests have been given to that URL, so after a while, I tried again with the difference that I added up to 10 seconds of delay and still gave me the code. (429)
Here's the code:
import requests
import time
query = "programming"
while True:
response = requests.get('https://www.google.com/search?q=' + query)
print (response.status_code)
time.sleep(10) # How much delay should I add?
output:
.
.
.
200
200
200
429
429
429
429
429
429
.
.
.
My question is that How much delay should I add to keep the code running all day (like 12 hours) with the response code 200?
Am I doing the ideal way? because I found something on the web about Quota and pricing for google and wanted to know does that relates to this situation.
You are using google end-user frontend (what humans see and use to search things on google). There is no documentation regarding its rate limits as I don't think they want you to use it programmatically.
In order to get search results programmatically I think you are suppoosed to use the Google Custom Search JSON API which provides 100 calls per day for free.