[SOLVED] Running Selenium webdriver from a Python CGI script

Running Selenium webdriver from a Python CGI script

I created a python script that uses Selenium webdriver to scrap a website. Now I'm trying to run this script from the web using CGI. So to ensure that my CGI server is working I tried this :

import cgi
print 'Content-Type: text/html'
print
list_brand = ['VOLVO','FIAT', 'BMW']
print '<h1>TESTING CGI</h1>'
print '<form>'
print '<select>'
for i in range(3):
      print '<option value="' + list_brand[i] + '">'+ list_brand[i] +'</option>'
print '</select>'
print '</form>'

And it worked fine. Now, When I use Selenium with CGI using this script:

import cgitb
import cgi
from selenium import webdriver

print 'Content-Type: text/html'
print
cgitb.enable(display=0, logdir="C:/path/to/log/directory")
path_to_pjs = 'C:path/to/phantomjs-2.1.1-windows/bin/phantomjs.exe'
browser = webdriver.PhantomJS(executable_path = path_to_pjs)
#Reaching to URL
url = 'http://www.website.fr/cl/2/products'
browser.get(url)
div_set = browser.find_elements_by_class_name('productname')
print '<form>'
print '<select>'
for div in div_set:
      print '<option value="' + div.find_element_vy_tag_name('h3').text + '">'+ div.find_element_vy_tag_name('h3').text +'</option>'
print '</select>'
print '</form>'

the page keeps loading but doesn't respond. Any idea if this is even possible (I mean running selenium from a cgi script) or why my server doesn't respond ?

Solution

Well, I found the solution for my problem! for one : I didn't pay attention that I wrote vy instead of by in my functions : div.find_element_by_tag_name. And the second thing was using an Apache server. For some reason the lite python server using CGIHTTPServer doesn't work. So I used XAMPP modified the httpd.conf file and the last thing was adding the path #!/Python27/python to the script.