Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Correct way to try/except using Python requests module?

try:     r = requests.get(url, params={'s': thing}) except requests.ConnectionError, e:     print e #should I also sys.exit(1) after this? 

Is this correct? Is there a better way to structure this? Will this cover all my bases?

like image 475
John Smith Avatar asked May 12 '13 19:05

John Smith


People also ask

How does Python handle try except?

In Python, exceptions can be handled using a try statement. The critical operation which can raise an exception is placed inside the try clause. The code that handles the exceptions is written in the except clause. We can thus choose what operations to perform once we have caught the exception.

How does requests module work in Python?

Definition and Usage. The requests module allows you to send HTTP requests using Python. The HTTP request returns a Response Object with all the response data (content, encoding, status, etc).

What are the types of exceptions and how will you handle it in Python?

Try and Except Statement - Catching Exceptions In Python, we catch exceptions and handle them using try and except code blocks. The try clause contains the code that can raise an exception, while the except clause contains the code lines that handle the exception.

What is Raise_for_status in Python?

raise_for_status() returns an HTTPError object if an error has occurred during the process. It is used for debugging the requests module and is an integral part of Python requests. Python requests are generally used to fetch the content from a particular resource URI.


2 Answers

Have a look at the Requests exception docs. In short:

In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception.

In the event of the rare invalid HTTP response, Requests will raise an HTTPError exception.

If a request times out, a Timeout exception is raised.

If a request exceeds the configured number of maximum redirections, a TooManyRedirects exception is raised.

All exceptions that Requests explicitly raises inherit from requests.exceptions.RequestException.

To answer your question, what you show will not cover all of your bases. You'll only catch connection-related errors, not ones that time out.

What to do when you catch the exception is really up to the design of your script/program. Is it acceptable to exit? Can you go on and try again? If the error is catastrophic and you can't go on, then yes, you may abort your program by raising SystemExit (a nice way to both print an error and call sys.exit).

You can either catch the base-class exception, which will handle all cases:

try:     r = requests.get(url, params={'s': thing}) except requests.exceptions.RequestException as e:  # This is the correct syntax     raise SystemExit(e) 

Or you can catch them separately and do different things.

try:     r = requests.get(url, params={'s': thing}) except requests.exceptions.Timeout:     # Maybe set up for a retry, or continue in a retry loop except requests.exceptions.TooManyRedirects:     # Tell the user their URL was bad and try a different one except requests.exceptions.RequestException as e:     # catastrophic error. bail.     raise SystemExit(e) 

As Christian pointed out:

If you want http errors (e.g. 401 Unauthorized) to raise exceptions, you can call Response.raise_for_status. That will raise an HTTPError, if the response was an http error.

An example:

try:     r = requests.get('http://www.google.com/nothere')     r.raise_for_status() except requests.exceptions.HTTPError as err:     raise SystemExit(err) 

Will print:

404 Client Error: Not Found for url: http://www.google.com/nothere 
like image 104
Jonathon Reinhart Avatar answered Oct 20 '22 23:10

Jonathon Reinhart


One additional suggestion to be explicit. It seems best to go from specific to general down the stack of errors to get the desired error to be caught, so the specific ones don't get masked by the general one.

url='http://www.google.com/blahblah'  try:     r = requests.get(url,timeout=3)     r.raise_for_status() except requests.exceptions.HTTPError as errh:     print ("Http Error:",errh) except requests.exceptions.ConnectionError as errc:     print ("Error Connecting:",errc) except requests.exceptions.Timeout as errt:     print ("Timeout Error:",errt) except requests.exceptions.RequestException as err:     print ("OOps: Something Else",err)  Http Error: 404 Client Error: Not Found for url: http://www.google.com/blahblah 

vs

url='http://www.google.com/blahblah'  try:     r = requests.get(url,timeout=3)     r.raise_for_status() except requests.exceptions.RequestException as err:     print ("OOps: Something Else",err) except requests.exceptions.HTTPError as errh:     print ("Http Error:",errh) except requests.exceptions.ConnectionError as errc:     print ("Error Connecting:",errc) except requests.exceptions.Timeout as errt:     print ("Timeout Error:",errt)       OOps: Something Else 404 Client Error: Not Found for url: http://www.google.com/blahblah 
like image 44
jouell Avatar answered Oct 21 '22 00:10

jouell