I've been all over the net tonight, for about 3-4 hours now. I've tried every suggestion I've come across. I've even checked the "capabilities" object on my Selenium driver object to ensure that it is actually set there, and indeed, it is:
#<Selenium::WebDriver::Remote::Capabilities:0x00000007475cf0
@capabilities=
{:browser_name=>"phantomjs",
:version=>"1.9.7",
:platform=>:"linux-unknown-64bit",
:javascript_enabled=>true,
:css_selectors_enabled=>true,
:takes_screenshot=>true,
:native_events=>true,
:rotatable=>false,
:firefox_profile=>nil,
:proxy=>#<Selenium::WebDriver::Proxy:0x00000007475908 @type=:direct>,
"driverName"=>"ghostdriver",
"driverVersion"=>"1.1.0",
"handlesAlerts"=>false,
"databaseEnabled"=>false,
"locationContextEnabled"=>false,
"applicationCacheEnabled"=>false,
"browserConnectionEnabled"=>false,
"webStorageEnabled"=>false,
"acceptSslCerts"=>false,
"proxy"=>{"proxyType"=>"direct"},
"phantomjs.page.settings.userAgent"=>
"Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:27.0) Gecko/20100101 Firefox/27.0"}>
Besides "phantomjs.page.settings.userAgent", I've tried "userAgent", etc. Every thing I could find online in the last 3-4 hours, I've tried. Apparently, near the beginning of 2013, this was a quite common question, and the solutions I'm speaking of were apparently the common solutions. None of these are working, and in fact, I know this for sure from this bit of information (note that User-Agent is "Ruby"):
UNCAUGHT EXCEPTION: {"errorMessage"=>"Element is not currently visible and may not be manipulated",
"request"=>
{"headers"=>
{"Accept"=>"application/json",
"Accept-Encoding"=>"gzip;q=1.0,deflate;q=0.6,identity;q=0.3",
"Connection"=>"close",
"Content-Length"=>"2",
"Content-Type"=>"application/x-www-form-urlencoded",
"Host"=>"localhost:9876",
"User-Agent"=>"Ruby"},
"httpVersion"=>"1.1",
"method"=>"POST",
"post"=>"{}",
"postRaw"=>"{}",
"url"=>"/click",
"urlParsed"=>
{"anchor"=>"",
"query"=>"",
"file"=>"click",
"directory"=>"/",
"path"=>"/click",
"relative"=>"/click",
"port"=>"",
"host"=>"",
"password"=>"",
"user"=>"",
"userInfo"=>"",
"authority"=>"",
"protocol"=>"",
"source"=>"/click",
"queryKey"=>{},
"chunks"=>["click"]},
"urlOriginal"=>
"/session/a03cc440-4f5c-11e4-8854-ed9c22bf60af/element/%3Awdc%3A1412822036214/click"}}
Unfortunately, there is a lot more information and discussion on these kinds of Selenium issues, and many others, if you're using Java. At this point, I wish in every way I had gone with Java for this project, but now I have 30,000 lines of code that I wrote entirely myself over the past 2 months. Losing this work, now at least, would not only be devastating for me personally, but would be disastrous for my job.
What gives? Am I really going to have to dig in and customize the source to get what I want, or is this feature really implemented now? Again, I saw all the answers from early 2013, but they don't work for me, and I have no idea why, or how to easily fix it. I'm on a deadline so this is beginning to be very stressful.
Does anyone have any ideas for me? Keep in mind I'm using Ruby, not Java.
Selenium-webdriver is 2.43. PhantomJS is 1.9.7. GhostDriver is 1.1.0.
This seems incredulous, to me, that there is no way to modify your User-Agent.
Please let me know if I can supply any other information that might be of assistance.
I appreciate in advance if you'd be so kind to share a few ideas or some information to get me pointed in the right direction.
This ended up not being "possible" out-of-the-box, as moonfly mentioned in his comments to my question. However, this turns out to be relatively easy to do. I should note, that this did not solve my particular problem. I assumed, perhaps naively, that I was getting strange results because of the User-Agent 'Ruby'. Turns out I spent hours figuring this out for nothing; I still get the same result.
But, for those of you that this will help, you'll have to make a quick monkey-patch to Selenium. Nasty, gross, ugly, hacky, I know. But, it did the trick.
Note, that this is a monkey-patch for Ruby's selenium-webdriver, version 2.43.0 -- if you have another version, there is no guarantee this will work.
module Selenium
module WebDriver
module Remote
module Http
class Default < Common
private
def request(verb, url, headers, payload, redirects = 0)
# THIS IS WHERE OUR CUSTOM CHANGE BEGINS
headers.merge!('User-Agent' => 'Whatever User Agent You May Want')
# THIS IS WHERE OUR CUSTOM CHANGE ENDS
request = new_request_for(verb, url, headers, payload)
retries = 0
begin
response = response_for(request)
rescue Errno::ECONNABORTED, Errno::ECONNRESET, Errno::EADDRINUSE
# a retry is sometimes needed on Windows XP where we may quickly
# run out of ephemeral ports
#
# A more robust solution is bumping the MaxUserPort setting
# as described here:
#
# http://msdn.microsoft.com/en-us/library/aa560610%28v=bts.20%29.aspx
raise if retries >= MAX_RETRIES
request = new_request_for(verb, url, headers, payload)
retries += 1
retry
rescue Errno::ECONNREFUSED => ex
if use_proxy?
raise ex.class, "using proxy: #{proxy.http}"
else
raise
end
end
if response.kind_of? Net::HTTPRedirection
raise Error::WebDriverError, "too many redirects" if redirects >= MAX_REDIRECTS
request(:get, URI.parse(response['Location']), DEFAULT_HEADERS.dup, nil, redirects + 1)
else
create_response response.code, response.body, response.content_type
end
end
end
end
end
end
end