How to get the exception string from requests.exceptions.RequestException

Question

I have the below flask code :

from flask import Flask,request,jsonify
import requests
from werkzeug.exceptions import InternalServerError, NotFound
import sys
import json



app = Flask(__name__)
app.config['SECRET_KEY'] = "Secret!"

class InvalidUsage(Exception):
    status_code = 400

    def __init__(self, message, status_code=None, payload=None):
        Exception.__init__(self)
        self.message = message
        if status_code is not None:
            self.status_code = status_code
        self.payload = payload

    def to_dict(self):
        rv = dict(self.payload or ())
        rv['message'] = self.message
        rv['status_code'] = self.status_code
        return rv

@app.errorhandler(InvalidUsage)
def handle_invalid_usage(error):
    response = jsonify(error.to_dict())
    response.status_code = error.status_code
    return response

@app.route('/test',methods=["GET","POST"])
def test():
    url = "https://httpbin.org/status/404"
    try:
        response = requests.get(url)
        if response.status_code != 200:
            try:
                response.raise_for_status()
            except requests.exceptions.HTTPError:
                status = response.status_code
                print status
                raise InvalidUsage("An HTTP exception has been raised",status_code=status)
    except requests.exceptions.RequestException as e:
        print e

if __name__ == "__main__":
    app.run(debug=True)

My question is how do i get the exception string(message) and other relevant params from the requests.exceptions.RequestException object e ?

Also what is the best way to log such exceptions . In case of an HTTPError exceptions i have the status code to refer to.

But requests.exceptions.RequestException catches all request exceptions . So how do i differentiate between them and also what is the best way to log them apart from using print statements.

Thanks a lot in advance for any answers.

Answer 1

RequestException is a base class for HTTPError, ConnectionError, Timeout, URLRequired, TooManyRedirects and others (the whole list is available at the GitHub page of requests module). Seems that the best way of dealing with each error and printing the corresponding information is by handling them starting from more specific and finishing with the most general one (the base class). This has been elaborated widely in the comments in this StackOverflow topic. For your test() method this could be:

@app.route('/test',methods=["GET","POST"])
def test():
    url = "https://httpbin.org/status/404"
    try:
        # some code...
    except requests.exceptions.ConnectionError as ece:
        print("Connection Error:", ece)
    except requests.exceptions.Timeout as et:
        print("Timeout Error:", et)
    except requests.exceptions.RequestException as e:
        print("Some Ambiguous Exception:", e)

This way you can firstly catch the errors that inherit from the RequestException class and which are more specific.

And considering an alternative for printing statements - I'm not sure if that's exactly what you meant, but you can log into console or to a file with standard Python logging in Flask or with the logging module itself (here for Python 3).

Answer 2

This is actually not a question about using the requests library as much as it is a general Python question about how to extract the error string from an exception instance. The answer is relatively straightforward: you convert it to a string by calling str() on the exception instance. Any properly written exception handler (in requests or otherwise) would have implemented an __str__() method to allow an str() call on an instance. Example below:

import requests

rsp = requests.get('https://httpbin.org/status/404')
try:
    if rsp.status_code >= 400:
        rsp.raise_for_status()
except requests.exceptions.RequestException as e:
    error_str = str(e)
    # log 'error_str' to disk, a database, etc.
    print('The error was:', error_str)

Yes, in this example, we print it, but once you have the string you have additional options. Anyway, saving this to test.py results in the following output given your test URL:

$ python3 test.py
The error was: 404 Client Error: NOT FOUND for url: https://httpbin.org/status/404