[SOLVED] Ruby Proxy Authentication GET/POST with OpenURI or net/http

Ruby Proxy Authentication GET/POST with OpenURI or net/http

I'm using ruby 1.9.3 and trying to use open-uri to get a url and try posting using Net:HTTP

Im trying to use proxy authentication for both:

Trying to do a POST request with net/http:

require 'net/http'
require 'open-uri'
http = Net::HTTP.new("google.com", 80)
headers = { 'User-Agent' => 'Ruby 193'}

resp, data = http.post("/", "name1=value1&name2=value2", headers)
puts data

And for open-uri which I can't get to do POST I use:

data = open("http://google.com/","User-Agent"=> "Ruby 193").read

How would I modify these to use a proxy with HTTP Authentication

I've tried (for open-uri)

data = open("http://google.com/","User-Agent"=> "Ruby 193", :proxy_http_basic_authentication => ["http://proxy.com:8000/", "proxy-user", "proxy-password"]).read

However all I will get is a OpenURI::HTTPError: 407 Proxy Authentication Required. I've verified all and it works in the browser with the same authentication and proxy details but I can't get ruby to do it.

How would I modify the code above to add http authentication properly? Has anyone gone through this atrocity?

Solution

Try:

require "open-uri"
proxy_uri = URI.parse("http://proxy.com:8000")

data = open("http://www.whatismyipaddress.com/", :proxy_http_basic_authentication => [proxy_uri, "username", "password"]).read
puts data

As for Net::HTTP, I recently implemented support for proxies with http authentication into a Net::HTTP wrapper library called http. If you look at my last pull-request, you'll see the basic implementation.

EDIT: Hopefully this will get you moving in the right direction.

Net::HTTP::Proxy(proxy_uri.host, proxy_uri.port,"username","password").start('whatismyipaddress.com') do |http| 
  puts http.get('/').body
end

EDIT 11/24/2020: Net::HTTP::Proxy is now considered obsolete. You can now configure proxies when creating a new instance of Net::HTTP. See the documentation for Net::HTTP.new for more details.