Python error: 'NoneType' object has no attribute 'find_all'

Question

I'm adapting a web scraping program from, http://danielfrg.com/blog/2013/04/01/nba-scraping-data/#disqus_thread, to scrape ESPN for baseball data into a CSV. However when I run the second piece of code to write a csv of games I get the 'NoneType' object has no attribute 'find_all' error, from the following section of code

for index, row in teams.iterrows():
    _team, url = row['team'], row['url']
    r = requests.get(BASE_URL.format(row['prefix_1'], year, row['prefix_2']))
    table = BeautifulSoup(r.text).table
    for row in table.find_all("tr")[1:]: # Remove header
        columns = row.find_all('td')
        try:
            _home = True if columns[1].li.text == 'vs' else False
            _other_team = columns[1].find_all('a')[1].text
            _score = columns[2].a.text.split(' ')[0].split('-')
            _won = True if columns[2].span.text == 'W' else False

            match_id.append(columns[2].a['href'].split('?id=')[1])
            home_team.append(_team if _home else _other_team)
            visit_team.append(_team if not _home else _other_team)
            d = datetime.strptime(columns[0].text, '%a, %b %d')
            dates.append(date(year, d.month, d.day))

I can post the whole program but this is the piece of code the compiler reads the error for.

The full error text is

Traceback (most recent call last):
  File "C:\Python27\Project Files\Game Parser.py", line 23, in <module>
    for row in table.find_all("tr")[1:]: # Remove header
AttributeError: 'NoneType' object has no attribute 'find_all'

Any help on how to get this code running would be greatly appreciated.

Answer 1

The error means that the table variable that you are building by doing:

table = BeautifulSoup(r.text).table

is returning None. And for row in table.find_all("tr")[1:]: on a None is throwing the error.

You can check if the url in question has a table in the way you are trying to access it. You can do this by printing out the url constructed by this statement:

BASE_URL.format(row['prefix_1'], year, row['prefix_2'])

and then going to this url in your browser to check if it has the table of your interest.