I am running selenium automation scripts using python3, on linux and undetected_chrome.
I am also using proxies.
When I run only one instances it works fine and does the job however the moment a second instance is launched I get this error :
2024-04-30 00:09:28 - ERROR - HTTPConnectionPool(host='localhost',
port=44073): Max retries exceeded with url:
/session/2488e939aa54121cb42b165948495489/actions (Caused by
NewConnectionError('<urllib3.connection.HTTPConnection object at
0x7debd7c86fb0>: Failed to establish a new connection: [Errno 111]
Connection refused'))
Despite the proxies are changing with each driver.
I have trying all kinds of solutions I could think of around it.
This solution uses SeleniumBase UC Mode instead of undetected-chromedriver
to spin up multiple simultaneous web browsers with stealth: (pip install seleniumbase
)
import sys
from concurrent.futures import ThreadPoolExecutor
from seleniumbase import Driver
sys.argv.append("-n") # Activate SB thread-locking as needed
def launch_driver(url):
driver = Driver(uc=True)
try:
driver.uc_open_with_reconnect(url, 4)
driver.sleep(2)
finally:
driver.quit()
urls = ['https://gitlab.com/users/sign_in' for i in range(3)]
with ThreadPoolExecutor(max_workers=len(urls)) as executor:
for url in urls:
executor.submit(launch_driver, url)
Be sure to swap the example URL (that uses bot-detection software) with the URL that you want.