Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to close urllib2 connection? [duplicate]

I have made a program using urllib2 that makes a lot of connections across the web. I noticed that eventually that this can be DDoS worthy; I would like to know how to close down each connection after I have done my business to prevent such an attack.

The code I am using to open a connection is:

cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
r = opener.open("http://www.python.org)
html = r.read()
like image 753
nobody Avatar asked Jul 05 '11 23:07

nobody


People also ask

Is urllib2 deprecated?

urllib2 is deprecated in python 3. x. use urllib instaed.

What is the difference between Urllib and urllib2?

1) urllib2 can accept a Request object to set the headers for a URL request, urllib accepts only a URL. 2) urllib provides the urlencode method which is used for the generation of GET query strings, urllib2 doesn't have such a function. This is one of the reasons why urllib is often used along with urllib2.

What does urllib2 Urlopen do?

The urllib2 module defines functions and classes which help in opening URLs (mostly HTTP) in a complex world — basic and digest authentication, redirections, cookies and more. Open the URL url, which can be either a string or a Request object. HTTPS requests do not do any verification of the server's certificate.

What does urlopen return?

The data returned by urlopen() or urlretrieve() is the raw data returned by the server. This may be binary data (such as an image), plain text or (for example) HTML. The HTTP protocol provides type information in the reply header, which can be inspected by looking at the Content-Type header.


2 Answers

I assume you are opening them with the urlopen() function. Its documentation states:

This function returns a file-like object with two additional methods:

As a file-like object, it will have a close method which you can call:

connection = urllib2.urlopen(url)
# Do cool stuff in here.
connection.close()

Update: Using the code you added to your question:

>>> import urllib2
>>> import cookielib
>>> cj = cookielib.CookieJar()
>>> opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
>>> r = opener.open("http://www.python.org")
>>> html = r.read()
>>> r.close??
Type:  instancemethod
Base Class: <type 'instancemethod'>
String Form: <bound method addinfourl.close of <addinfourl at 150857644 whose fp = <socket._fileobject object at 0x8fd48ec>>>
Namespace: Interactive
File:  /usr/lib/python2.6/urllib.py
Definition: r.close(self)
Source:
    def close(self):
        self.read = None
        self.readline = None
        self.readlines = None
        self.fileno = None
        if self.fp: self.fp.close()
        self.fp = None

So the close() method exists and actually does something:

>>> r.close()
>>> r.read()
------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython console>", line 1, in <module>
TypeError: 'NoneType' object is not callable
like image 137
Blair Avatar answered Sep 22 '22 12:09

Blair


your question is extremely vague.

but here is an example of closing a connection after use:

f = urllib2.urlopen(req)
f.read()
f.close()
like image 23
Corey Goldberg Avatar answered Sep 22 '22 12:09

Corey Goldberg