Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

closing files properly opened with urllib2.urlopen()

I have following code in a python script

  try:
    # send the query request
    sf = urllib2.urlopen(search_query)
    search_soup = BeautifulSoup.BeautifulStoneSoup(sf.read())
    sf.close()
  except Exception, err:
    print("Couldn't get programme information.")
    print(str(err))
    return

I'm concerned because if I encounter an error on sf.read(), then sf.clsoe() is not called. I tried putting sf.close() in a finally block, but if there's an exception on urlopen() then there's no file to close and I encounter an exception in the finally block!

So then I tried

  try:
    with urllib2.urlopen(search_query) as sf:
      search_soup = BeautifulSoup.BeautifulStoneSoup(sf.read())
  except Exception, err:
    print("Couldn't get programme information.")
    print(str(err))
    return

but this raised a invalid syntax error on the with... line. How can I best handle this, I feel stupid!

As commenters have pointed out, I am using Pys60 which is python 2.5.4

like image 795
fearoffours Avatar asked Oct 07 '10 10:10

fearoffours


People also ask

What does urllib2 Urlopen do?

urllib2 offers a very simple interface, in the form of the urlopen function. Just pass the URL to urlopen() to get a “file-like” handle to the remote data. like basic authentication, cookies, proxies and so on. These are provided by objects called handlers and openers.

What does Urllib request Urlopen do?

request is a Python module for fetching URLs (Uniform Resource Locators). It offers a very simple interface, in the form of the urlopen function. This is capable of fetching URLs using a variety of different protocols.

Which is better Urllib or requests?

Requests - Requests' is a simple, easy-to-use HTTP library written in Python. 1) Python Requests encodes the parameters automatically so you just pass them as simple arguments, unlike in the case of urllib, where you need to use the method urllib. encode() to encode the parameters before passing them.

Does urllib2 work in Python 3?

The urllib2 module supported a lot of functions and classes that helped the users to open an URL and extract the contents. However, In Python3, the urllib2 module is not available. Instead, the module is split into several sub-modules like urllib. request , urllib.


2 Answers

I would use contextlib.closing (in combination with from __future__ import with_statement for old Python versions):

from contextlib import closing

with closing(urllib2.urlopen('http://blah')) as sf:
    search_soup = BeautifulSoup.BeautifulStoneSoup(sf.read())

Or, if you want to avoid the with statement:

try:
    sf = None
    sf = urllib2.urlopen('http://blah')
    search_soup = BeautifulSoup.BeautifulStoneSoup(sf.read())
finally:
    if sf:
        sf.close()

Not quite as elegant though.

like image 95
Daniel Avatar answered Nov 03 '22 12:11

Daniel


finally:
    if sf: sf.close()
like image 40
dgarcia Avatar answered Nov 03 '22 10:11

dgarcia