Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

BeautifulSoup - TypeError: 'NoneType' object is not callable

I need to make my code backwards compatible with python2.6 and BeautifulSoup 3. My code was written using python2.7 and at this case using BS4. But when I try to run it at squeezy server, I get this error (it has python2.6 and bs3):

try:
    from bs4 import BeautifulSoup
except ImportError:
    from BeautifulSoup import BeautifulSoup

gmp = open(fname, 'r')
soup = BeautifulSoup(gmp)
p = soup.body.div.find_all('p')

p = soup.body.div.find_all('p')
TypeError: 'NoneType' object is not callable

If I change to:

   p = soup.body.div.findAll('p')

then I get this error:

p = soup.body.div.findAll('p')
TypeError: 'NoneType' object is not callable

Update of thrown error

  File "/home/user/openerp/7.0/addons/my_module/models/gec.py", line 401, in parse_html_data
    p = soup.body.div.findAll('p') #used findAll instead of find_all for backwards compatability to bs3 version
TypeError: 'NoneType' object is not callable

Either way, both approaches work on my Ubuntu with python2.7 and bs4, but not on squeezy. Is there any other difference between those versions that I don't see/know and gives me this error?

like image 599
Andrius Avatar asked Oct 28 '14 08:10

Andrius


1 Answers

You are using BeautifulSoup 3, but are using BeautifulSoup 4 syntax.

Your fallback is at fault here:

try:
    from bs4 import BeautifulSoup
except ImportError:
    from BeautifulSoup import BeautifulSoup

If you want to use either version 3 or 4, stick to version 3 syntax:

p = soup.body.div.findAll('p')

because find_all is not a valid method in BeautifulSoup 3, so it is instead interpreted as a tag search. There is no find_all tag in your HTML, so None is returned, which you then try to call.

Next, the parser used by BeautifulSoup 3 will respond differently to broken or incomplete HTML. If you have lxml installed on Ubuntu, then that'll be used as the default parser, and it'll insert a missing <body> tag for you. BeautifulSoup 3 may leave that out.

I strongly urge you to instead remove the fallback, and stick with BeautifulSoup version 4 only. Version 3 has been discontinued years ago, and contains unfixed bugs. BeautifulSoup 4 also offers additional features you may want to make use of.

BeautifulSoup is pure Python and easily installed into a virtual environment on any platform supported by Python. You are not tied to the system-supplied package here.

On Debian Squeezy for example, you'd be stuck with BeautifulSoup 3.1.0, and even the BeautifulSoup developers do not want you to use it!. Your problem with findAll almost certainly stems from using that release.

like image 187
Martijn Pieters Avatar answered Nov 15 '22 19:11

Martijn Pieters