Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I get pytest to not catch exceptions

When I run pytest in the vscode debugger with "Uncaught Exceptions" checked, and there are test errors, no uncaught exceptions occur, because pytest catches them do its results reporting. How can I tell pytest to just let the exceptions happen? So that I can catch them in the vscode debugger?

Basically I want behavior like --pdb but I want it to launch the vscode debugger not pdb. The flag --pdbcls sounds promising but not sure what <module>:<class> to give it.

Note: Normally I would just have it break on Raised Exceptions. But our code has tons of raised-but-caught exceptions so this option is not useful.

Here's a video of vscode not breaking when an AssertionError fires while debugging a pytest test:

enter image description here

@rioV8's suggestion below does break the on the exception, but for some reason there is no stack which means you can't debug from there:

enter image description here

I must be missing something because no one else seems to need this capability. But to me this seems like absolutely the most basic simplest thing one could do with a testing framework and a debugger: as a developer I want to debug from the point where the error is raised.

There must be some totally other way people are using a debugger with pytest, some obvious technique I'm overlooking.

like image 983
Philip Avatar asked Jun 17 '20 01:06

Philip


People also ask

How do you test if an exception is caught Python?

What you can do is split the code that raises the exception into its own function, and then test that to check that the exception is raised (with assertRaises ). The original function can call that one, and if your normal tests on that work then apparently it catches the exception.

How do I catch exceptions in Visual Studio?

You can also add or delete exceptions. With a solution open in Visual Studio, use Debug > Windows > Exception Settings to open the Exception Settings window. Provide handlers that respond to the most important exceptions.


2 Answers

I raised an issue with pytest and they made a suggestion that works. Add this to your conftest.py:

import os
import pytest

if os.getenv('_PYTEST_RAISE', "0") != "0":

    @pytest.hookimpl(tryfirst=True)
    def pytest_exception_interact(call):
        raise call.excinfo.value

    @pytest.hookimpl(tryfirst=True)
    def pytest_internalerror(excinfo):
        raise excinfo.value

Then in your "request": "test" launch.json you can toggle that env var:

    {
        "name": "Debug Tests",
        "type": "python",
        "request": "test",
        "console": "integratedTerminal",
        "justMyCode": true,
        "env": {
            "_PYTEST_RAISE": "1"
        },
    }

If _PYTEST_RAISE is set and you check only the Uncaught Exceptions box you will break right where the uncaught exception was raised not before.

I also opened an issue with debugpy and they had some ideas, but no solution today. The _PYTEST_RAISE trick works 100% solves this for me.

like image 91
Philip Avatar answered Oct 10 '22 20:10

Philip


I've noticed a few things that might be relevant. Even though I'm not ready to dig deeper right now, I'll post them here in case they might be useful to someone.

Workaround:

A simple hack I've used is to just (temporarily) call the relevant test case directly and "Run and Debug" the file rather than using the "Debug Test Case" button:


def test_example():
    assert False


test_example()

Result:

enter image description here

Notes:

  1. The important difference in behavior correlates with specifying "request": "test" vs specifying "request": "launch" in the launch.json configuration. (There is also a difference between the two types in whether or not the justMyCode option gets applied from a global launch configuration as described here).
  2. If you have a python "request": "test" specified, then VSCode doesn't appear to let you debug the file in launch mode as I did above.
  3. In my scenario, I had a test failing because of an exception I was causing in a third-party library (my fault, of course). Breakpoints in third-party code are apparently only triggered if the "justMyCode": false option is added.

launch.json:

{
    "version": "0.2.0",
    "configurations": [
        
        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "justMyCode": false,            
        },
        // {
        //     "name": "Debug Unit Test",
        //     "type": "python",
        //     "request": "test",
        //     "console": "integratedTerminal",
        //     "justMyCode": false,
        // }        
    ]
}
like image 31
teichert Avatar answered Oct 10 '22 22:10

teichert