pythonseleniumproxygeckodriverfoxyproxy

Cannot use Private Socks5 proxy with Selenium+Firefox (Python)


Context: I am using Python, selenium + Geckodriver. Firefox webbrowser automation and I have problem on a private proxy authentification. I cannot manage to configure my socks5 proxy username and password. Thus, Firefox shows up but connection doesnt work.

The simple task that I want to do is :

  1. I want to use selenium(automated Firefox) with a proxy (SOCKS5 proxy with password)
  2. Checking my IP on a website to see if the proxy is working

What I noted so far is:

So the current code is, replace HOST/USERNAME/PORT/PASSWORD with socks5 proxy data :

    from selenium import webdriver
    from base64 import b64encode
    from selenium import webdriver

    def login(browser):
        alert=browser.switch_to_alert()
        alert.send_keys("username"+webdriver.common.keys.Keys.TAB+"password")
        alert.accept()

    proxy = {'host': HOST, 'port': PORT, 'usr': USERNAME, 'pwd': PASSWORD}
    fp = webdriver.FirefoxProfile()
    fp.set_preference('network.proxy.type', 1)
    fp.set_preference('network.proxy.http', proxy['host'])
    fp.set_preference('network.proxy.http_port', int(proxy['port']))
    fp.set_preference('network.proxy.no_proxies_on', 'localhost, 127.0.0.1')
    fp.set_preference("network.proxy.socks_version", 5)
    credentials = '{usr}:{pwd}'.format(**proxy)
    fp.set_preference("network.proxy.type", 1)
    fp.set_preference("network.proxy.socks_version", 5)
    fp.set_preference("network.proxy.socks", HOST)
    fp.set_preference("network.proxy.socks_port", PORT)
    fp.set_preference("network.http.use-cache", False)
    driver = webdriver.Firefox(fp)
    # normally a proxy login popup should appear and we should call the login function here!!
    driver.get('https://wtfismyip.com')

Doesnt work, Any Ideas ?


Solution

  • To add foxyproxy as an extension:

    driver = webdriver.Firefox(executable_path="path to geckodriver", options=options)
    driver.install_addon("path to .xpi file of the extension", temporary=None)
    driver.switch_to.window(driver.window_handles[1])`
    

    So it should open a webpage where you can set your proxy. Now, to set the proxy, first i define a function that lets me insert all the proxy's stuff (host, port, username, password)

    def set_proxy(address, port, usernamex, passwordx):
        proxy_address_xpath = '//*[@id="proxyAddress"]'
        proxy_address_box = driver.find_element_by_xpath(proxy_address_xpath)
        proxy_address_box.send_keys(address)
        proxy_port_xpath = '//*[@id="proxyPort"]'
        proxy_port_box = driver.find_element_by_xpath(proxy_port_xpath)
        proxy_port_box.send_keys(port)
        proxy_username_xpath = '//*[@id="proxyUsername"]'
        proxy_username_box = driver.find_element_by_xpath(proxy_username_xpath)
        proxy_username_box.send_keys(usernamex)
        proxy_pass_xpath = '//*[@id="proxyPassword"]'
        proxy_pass_box = driver.find_element_by_xpath(proxy_pass_xpath)
        proxy_pass_box.send_keys(passwordx)
        save_xpath = '/html/body/div[2]/div[2]/button[4]'
        save_box = driver.find_element_by_xpath(save_xpath)
        save_box.click()
    

    then, with the following code, I set the proxy and select it, so that when i navigate it will use the proxy

    # go to the page where you can add the proxies
    xpath_back ="/html/body/div[2]/div[2]/button"
    back_box = driver.find_element_by_xpath(xpath_back)
    back_box.click()
        
    # add proxy
    add_xpath = "/html/body/div[4]/div/nav/a[1]"
    add_box = WebDriverWait(driver, timeout=10).until(lambda x: x.find_element_by_xpath(add_xpath))
    add_box.click()
        
    # select proxy type (in my case socks5)
    type_xpath = '//*[@id="proxyType"]'
    type_box = driver.find_element_by_xpath(type_xpath)
    type_box.click()
        
    socks5_xpath = '/html/body/div[2]/div[1]/div[2]/div[1]/select/option[3]'
    socks5_box = driver.find_element_by_xpath(socks5_xpath)
    socks5_box.click()
        
    # write the proxy stuff
    set_proxy('socks5', 'host', 'port', 'pass')
        
    # select the proxy you just saved (in this case it selects the first one)
    change_xpath = '//*[@id="mode"]'
    change_box = WebDriverWait(driver, timeout=10).until(lambda x: x.find_element_by_xpath(change_xpath))
    change_box.click()
        
    select_xpath = '/html/body/div[4]/div/div/div[1]/select/option[2]'
    select_box = driver.find_element_by_xpath(select_xpath)
    select_box.click()
    

    then, to navigate, be sure to do that in a new windows or switch to the first one using

    driver.switch_to.window(driver.window_handles[0])
    

    so that you can keep the foxyproxy window for example to change proxy.

    If you want to use other ways, check How to set proxy authentication (user & password) using Python + Selenium