I'm trying to scrape some information from webpages that are inconsistent about where the info is located. I've got code to handle each of several possibilities; what I want is to try them in sequence, then if none of them work I'd like to fail gracefully and move on.
That is, in psuedo-code:
try:
info = look_in_first_place()
otherwise try:
info = look in_second_place()
otherwise try:
info = look_in_third_place()
except AttributeError:
info = "Info not found"
I could do this with nested try statements, but if I need 15 possibilities to try then I'll need 15 levels of indentation!
This seems like a trivial enough question that I feel like I'm missing something, but I've searched it into the ground and can't find anything that looks equivalent to this situation. Is there a sensible and Pythonic way to do this?
EDIT: As John's (pretty good) solution below raises, for brevity I've written each lookup above as a single function call, whereas in reality it's usually a small block of BeautifulSoup calls such as soup.find('h1', class_='parselikeHeader')
. Of course I could wrap these in functions, but it seems a bit inelegant with such simple blocks -- apologies if my shorthand changes the problem though.
This may be a more useful illustration:
try:
info = soup.find('h1', class_='parselikeHeader').get('href')
if that fails try:
marker = soup.find('span', class_='header')
info = '_'.join(marker.stripped_strings)
if that fails try:
(other options)
except AttributeError:
info = "Info not found"
If each lookup is a separate function, you can store all the functions in a list and then iterate over them one by one.
lookups = [
look_in_first_place,
look_in_second_place,
look_in_third_place
]
info = None
for lookup in lookups:
try:
info = lookup()
# exit the loop on success
break
except AttributeError:
# repeat the loop on failure
continue
# when the loop is finished, check if we found a result or not
if info:
# success
else:
# failure
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With