I Programmed this code to get website content but there is a problem when connection is filed the program stop and dont try to reconnect
url= 'https://website.com'
def get_page_content(url, head):
"""
Function to get the page content
"""
req = Request(url, headers=head)
return urlopen(req)
head = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Safari/537.36',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3',
'Accept-Encoding': 'none',
'Accept-Language': 'en-US,en;q=0.8',
'Connection': 'keep-alive',
'refere': 'https://example.com',
'cookie': """your cookie value ( you can get that from your web page) """
}
data = get_page_content(url, head).read()
I searched in the internet but didn't find solution.
Python has try, except blocks you can add. Basically you put the code that's failing (specifically the lines that throws errors) into the try section, and then the except portion will contain code that you want to do in the event of an error. You can use this with a flag in a loop to keep trying to connect a certain number of times
success_flag = False
while success_flag == False: # keep looping until success
try: # code in here will attempt to run
req = Request(url, headers=head)
page = urlopen(req)
success_flag = True # mark as successful run if got through the code that may fail
return page
except Exception as e: # this code will run in the event of a failure
print(e) # just print the exception before trying again.
# possibly change something before reconnecting depending on error thrown
keep in mind you don't have the traceback posted, so there may be an issue with the code itself. This code will just do as you asked and that is keep trying again. You may need to change something before trying a reconnect