pytest: When using the keyword option (-k) a directory style string is not handled

Question

I am currently using pytest to automate my test suite. I have tests spread over several directories like below:

|-Root
|   |-Dir1
|   |   |-Test1
|   |   |-Test2
|   |   |-Test3
|   |-Dir2
|   |   |-Testa
|   |   |-Testb
|   |   |-Testc
|   |-Dir3
|   |   |-TestI
|   |   |-TestII
|   |   |-TestIII

I would like to be able to run tests in a set of directories by excluding others. Some of the directories have similar names like "test_set_1" and "test_set_1_extended". I'd like to exclude "test_set_1" but keep "test_set_1_extended" so I'd need to use the full path as a keyword. I've noticed when using the keywords (-k option) that pytest doesn't handle directory structure strings as an argument well. For instance, if I try windows style

py.test -k "not (root\dir1 or dir2)"

I get a syntax error:

SyntaxError: unexpected character after line continuation character

On the other hand if I try unix style

py.test -k "not (root/dir1 or dir2)" 

I get a ZeroDivisionError:

ZeroDivisionError: integer division or modulo by zero

Things I've tried include: Enclosing the directory string in single or double quotes:

py.test -k "not ('root/dir1' or dir2)"
py.test -k "not ("root/dir1' or dir2)"

Using separate '-k's':

py.test -k "not root/dir1" -k not dir2)

Using a current directory '.'

py.test -k "not (./root/dir1 or dir2)"

Escaping the slash:

py.test -k "not (root\\dir1 or dir2)"
py.test -k "not (root\/dir1 or dir2)"

None of these have solved the problem. Does anyone have any suggestions on how to get pytest to handle my path string correctly?

Answer 1

Why not save yourself the workarounds and just use unique markers for your different test groups, and use

py.test -m "not extended" 

This way test/directory/module names can change and nothing breaks.

Answer 2

I am working with a very similar setup and I have been able to reproduce the behavior you are describing (pytest 3.4.0, python 3.6.4).

I have found a workaround that might be useful to you depending on your exact layout). Basically, I am using and instead of the path separator.

Therefore py.test -k "not (root/dir1 or dir2)" becomes py.test -k "not ((root and dir1) or dir2)". If you do not have any more dir1 path elements under root/dir1, both expressions should be equivalent.

Since the behavior of expressions including / seems somewhat inconsistent (py.test -k root/dir1 works as expected), it might make sense to file a bug.