opening websites using urllib2 from behind corporate firewall - 11004 getaddrinfo failed

Question

I am trying to access a website from behind corporate firewall using below:-

password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_mgr.add_password(None, url, username, password)
auth_handler = urllib2.HTTPBasicAuthHandler(password_mgr)
opener = urllib2.build_opener(auth_handler) 
urllib2.install_opener(opener) 
conn = urllib2.urlopen('http://python.org')

Getting error

URLError: <urlopen error [Errno 11004] getaddrinfo failed>

I have tried with different handlers (tried ProxyHandler also in slightly different way), but doesn't seem to work.

Any clues to what could be the reason for error and any different ways to supply the credentials and make it work?

Answer 1

If you are using Proxy and that proxy has Username and Password (which many corporate proxies have), you need to set the proxy handler with urllib2.

  proxy_url = 'http://' + proxy_user + ':' + proxy_password + '@' + proxy_ip
  proxy_support = urllib2.ProxyHandler({"http":proxy_url})
  opener = urllib2.build_opener(proxy_support,urllib2.HTTPHandler)
  urllib2.install_opener(opener)

HTTPBasicAuthHandler is used to provide credentials for the site which you are going to access and not for going through the proxy. The above snippet might help you.

Answer 2

On Windows, I observed that python uses the IE Internet Options-> LAN Settings settings. So even if we use urllib2 to install opener and specify the proxy_url, it would continue to use the IE settings.

It worked fine finally, when I exported a system variable:

http_proxy=http://userid:[email protected]:port