Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Proxy with urllib2

I open urls with:

site = urllib2.urlopen('http://google.com')

And what I want to do is connect the same way with a proxy I got somewhere telling me:

site = urllib2.urlopen('http://google.com', proxies={'http':'127.0.0.1'})

but that didn't work either.

I know urllib2 has something like a proxy handler, but I can't recall that function.

like image 552
Chris Stryker Avatar asked Sep 20 '09 02:09

Chris Stryker


People also ask

Is urllib2 deprecated?

urllib2 is deprecated in python 3. x. use urllib instaed.

Can I use urllib2 in python3?

NOTE: urllib2 is no longer available in Python 3 You can get more idea about urllib.

How do I use urllib2?

urllib2 offers a very simple interface, in the form of the urlopen function. Just pass the URL to urlopen() to get a “file-like” handle to the remote data. like basic authentication, cookies, proxies and so on. These are provided by objects called handlers and openers.

What does Urllib request Urlretrieve do?

In line 14, the urllib. request. urlretrieve() function is used to retrieve the image from the given url and store it to the required file directory.


2 Answers

proxy = urllib2.ProxyHandler({'http': '127.0.0.1'}) opener = urllib2.build_opener(proxy) urllib2.install_opener(opener) urllib2.urlopen('http://www.google.com') 
like image 169
ZelluX Avatar answered Sep 26 '22 03:09

ZelluX


You have to install a ProxyHandler

urllib2.install_opener(     urllib2.build_opener(         urllib2.ProxyHandler({'http': '127.0.0.1'})     ) ) urllib2.urlopen('http://www.google.com') 
like image 20
dcrosta Avatar answered Sep 23 '22 03:09

dcrosta