Stop at exception in my, not library code

Question

I'm developing an app using a Python library urllib and it is sometimes rising exceptions due to not being able to access an URL.

However, the exception is raised almost 6 levels into the standard library stack:

/home/user/Workspace/application/main.py in call(path)
     11                                  headers={'content-type': 'application/json'},
     12                                  data=b'')
---> 13     resp = urllib.request.urlopen(req)          ####### THIS IS MY CODE
     14     return json.loads(resp.read().decode('utf-8'))

/usr/lib/python3.4/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    159     else:
    160         opener = _opener
--> 161     return opener.open(url, data, timeout)
    162 
    163 def install_opener(opener):

/usr/lib/python3.4/urllib/request.py in open(self, fullurl, data, timeout)
    461             req = meth(req)
    462 
--> 463         response = self._open(req, data)
    464 
    465         # post-process response

/usr/lib/python3.4/urllib/request.py in _open(self, req, data)
    479         protocol = req.type
    480         result = self._call_chain(self.handle_open, protocol, protocol +
--> 481                                   '_open', req)
    482         if result:
    483             return result

/usr/lib/python3.4/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
    439         for handler in handlers:
    440             func = getattr(handler, meth_name)
--> 441             result = func(*args)
    442             if result is not None:
    443                 return result

/usr/lib/python3.4/urllib/request.py in http_open(self, req)
   1208 
   1209     def http_open(self, req):
-> 1210         return self.do_open(http.client.HTTPConnection, req)
   1211 
   1212     http_request = AbstractHTTPHandler.do_request_

/usr/lib/python3.4/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1182                 h.request(req.get_method(), req.selector, req.data, headers)
   1183             except OSError as err: # timeout error
-> 1184                 raise URLError(err)
   1185             r = h.getresponse()
   1186         except:

URLError: <urlopen error [Errno 111] Connection refused>

I usually run the code in ipython3 with the %pdb magic turned on so in case there is an exception I can inspect it immediately. However for this I have to go down the stack 6 levels to get to my code.

Is it achievable that my app crashes pointing to my code directly?

Answer 1

I would go with modifying the code:

try:
    resp = urllib.request.urlopen(req)

except Exception as e:
    raise RuntimeError(e)

That way:

• %pdb moves you to your code,
• the original exception is preserved as argument of the "secondary" exception.

You may also monkeypatch urllib.request.urlopen() function:

class MonkeyPatchUrllib(object):
    def __enter__(self):
        self.__urlopen = urllib.request.urlopen
        urllib.request.urlopen = self
    def __exit__(self, exception_type, exception_value, traceback):
        urllib.request.urlopen = self.__urlopen
    def __call__(self, *args, **kwargs):
        try:                                  
            return self.__urlopen(*args, **kwargs)
        except Exception as e:
            raise RuntimeError(e)

Any time you have an exception raised in urlibopen() call within the context manager scope:

with MonkeyPatchUrllib():
    #your code here

%pdb will move you only 1 level away from your code.

[EDIT]

With sys.exc_info() it is possible to preserve a more verbose context of the original exception (like its traceback).