I'm running a program that uses webbot in Python 3.x - the job of the program I'm writing, amongst other things, is to look for certain text patterns in a page full of text, and generate results based on if it finds a match or not. So I have a bunch of lines that look kind of like this:
if web.exists("hi I'm some text", loose_match=False) == True:
ifoundthetext = 1
else:
ifoundthetext = 0
This DOES work, however it takes about a second or two per individual search. I have a lot of different texts I need to search, and by the time it gets through all of them the program is at about the 10 second mark per cycle of the program. To make matters worse the longer I leave the program running (especially the compiled .exe version), the slower it gets (up to over a minute if the program has been running overnight). Yes I am using garbage collection.
I've tried combining text results using "or" statements, like this:
if web.exists("hi I'm some text", loose_match=False) == True or web.exists("hi I'm some other text", loose_match=False) == True:
But this has no effect on the program's speed. How can I make this thing go a bit quicker, but without ditching webbot?
Comments have made it clear that the deficiency is in Webbot itself, which is just too slow and not up to the task of all that repetitive searching.
I've reluctantly rewritten the entire program in Selenium, but I'm glad I did as it performs much faster!
Also used booleans just because of neatness.
The equivalent code in my Selenium version:
try:
driver.find_element_by_partial_link_text("hi I'm some text")
ifoundthetext = True
print ('yay')
except:
ifoundthetext = False