Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I get PyCharm to show entire error diffs from pytest?

I am using Pycharm to run my pytest unit tests. I am testing a REST API, so I often have to validate blocks of JSON. When a test fails, I'll see something like this:

FAILED test_document_api.py:0 (test_create_documents) {'items': [{'i...ages': 1, ...} != {'items': [{'...ages': 1, ...}  Expected :{'items': [{'...ages': 1, ...} Actual   :{'items': [{'i...ages': 1, ...}  <Click to see difference> 

When I click on the "Click to see difference" link, most of the difference is converted to points of ellipses, like so

Pycharm comparison with differences elided

This is useless since it doesn't show me what is different. I get this behavior for any difference larger than a single string or number.

I assume Pycharm and/or pytest tries to elide uninformative parts of differences for large outputs. However, it's being too aggressive here and eliding everything.

How do I get Pycharm and/or pytest to show me the entire difference?

I've tried adding -vvv to pytest's Additional Arguments, but that has no effect.


Since the original post I verified that I see the same behavior when I run unit tests from the command line. So this is an issue with pytest and not Pycharm.

After looking at the answers I've got so far I guess what I'm really asking is "in pytest is it possible to set maxDiff=None without changing the source code of your tests?" The impression I've gotten from reading about pytest is that the -vv switch is what controls this setting, but this does not appear to be the case.

like image 948
W.P. McNeill Avatar asked May 30 '18 15:05

W.P. McNeill


People also ask

How does PyCharm run all Pytests?

Run all tests in a directoryFrom the context menu, select the corresponding run command. If the directory contains tests that belong to the different testing frameworks, select the configuration to be used. For example, select Run pytest in <directory name>'. Explore results in the test runner.

How do I run a single test case of pytest in PyCharm?

PyCharm makes it easy to select just one test to run. In fact, there are several ways to do it: With the cursor anywhere in the test you want to focus on, right-click and choose to run that in the test runner. Right-click on the test in the test tool listing and choose to run it.

How do I set the default test runner in PyCharm?

To set a test runner, press Ctrl+Alt+S to open the IDE settings and select Tools | Python Integrated Tools, and then select the target test runner from the Default test runner list.


Video Answer


1 Answers

If you look closely into PyCharm sources, from the whole pytest output, PyCharm uses a single line the to parse the data for displaying in the Click to see difference dialog. This is the AssertionError: <message> line:

def test_spam(): >       assert v1 == v2 E       AssertionError: assert {'foo': 'bar'} == {'foo': 'baz'} E         Differing items: E         {'foo': 'bar'} != {'foo': 'baz'} E         Use -v to get the full diff 

If you want to see the full diff line without truncation, you need to customize this line in the output. For a single test, this can be done by adding a custom message to the assert statement:

def test_eggs():     assert a == b, '{0} != {1}'.format(a, b) 

If you want to apply this behaviour to all tests, define custom pytest_assertrepr_compare hook. In the conftest.py file:

# conftest.py def pytest_assertrepr_compare(config, op, left, right):     if op in ('==', '!='):         return ['{0} {1} {2}'.format(left, op, right)] 

The equality comparison of the values will now still be stripped when too long; to show the complete line, you still need to increase the verbosity with -vv flag.

Now the equality comparison of the values in the AssertionError line will not be stripped and the full diff is displayed in the Click to see difference dialog, highlighting the diff parts:

enter image description here

like image 65
hoefling Avatar answered Oct 05 '22 03:10

hoefling