Menu
I'm new to Python and reading someone else's code:
should urllib.urlopen() be followed by urllib.close()? Otherwise, one would leak connections, correct?
692
request is a Python module for fetching URLs (Uniform Resource Locators). It offers a very simple interface, in the form of the urlopen function. This is capable of fetching URLs using a variety of different protocols.
The data returned by urlopen() or urlretrieve() is the raw data returned by the server. This may be binary data (such as an image), plain text or (for example) HTML.
True, if you want to avoid adding any dependencies, urllib is available. But note that even the Python official documentation recommends the requests library: "The Requests package is recommended for a higher-level HTTP client interface."
I found that time took to send the data from the client to the server took same time for both modules (urllib, requests) but the time it took to return data from the server to the client is more then twice faster in urllib compare to request.
The close method must be called on the result of urllib.urlopen, not on the urllib module itself as you're thinking about (as you mention urllib.close -- which doesn't exist).
The best approach: instead of x = urllib.urlopen(u) etc, use:
import contextlib with contextlib.closing(urllib.urlopen(u)) as x: ...use x at will here... The with statement, and the closing context manager, will ensure proper closure even in presence of exceptions.
79
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
