Is there a way to control how pytest-xdist runs tests in parallel?

Question

I have the following directory layout:

runner.py lib/ tests/       testsuite1/                  testsuite1.py       testsuite2/                  testsuite2.py       testsuite3/                  testsuite3.py       testsuite4/                  testsuite4.py 

The format of testsuite*.py modules is as follows:

 import pytest  class testsomething:       def setup_class(self):           ''' do some setup '''           # Do some setup stuff here             def teardown_class(self):           '''' do some teardown'''           # Do some teardown stuff here        def test1(self):           # Do some test1 related stuff        def test2(self):           # Do some test2 related stuff        ....       ....       ....       def test40(self):           # Do some test40 related stuff  if __name__=='__main()__'    pytest.main(args=[os.path.abspath(__file__)]) 

The problem I have is that I would like to execute the 'testsuites' in parallel i.e. I want testsuite1, testsuite2, testsuite3 and testsuite4 to start execution in parallel but individual tests within the testsuites need to be executed serially.

When I use the 'xdist' plugin from py.test and kick off the tests using 'py.test -n 4', py.test is gathering all the tests and randomly load balancing the tests among 4 workers. This leads to the 'setup_class' method to be executed every time of each test within a 'testsuitex.py' module (which defeats my purpose. I want setup_class to be executed only once per class and tests executed serially there after).

Essentially what I want the execution to look like is:

 worker1: executes all tests in testsuite1.py serially worker2: executes all tests in testsuite2.py serially worker3: executes all tests in testsuite3.py serially worker4: executes all tests in testsuite4.py serially 

while worker1, worker2, worker3 and worker4 are all executed in parallel.

Is there a way to achieve this in 'pytest-xidst' framework?

The only option that I can think of is to kick off different processes to execute each test suite individually within runner.py:

  def test_execute_func(testsuite_path):     subprocess.process('py.test %s' % testsuite_path)  if __name__=='__main__':    #Gather all the testsuite names    for each testsuite:        multiprocessing.Process(test_execute_func,(testsuite_path,)) 

Answer 1

You can use --dist=loadscope to group all the tests in the same test class. Here is the doc from pytest-xdist on pypi

By default, the -n option will send pending tests to any worker that is available, without any guaranteed order, but you can control this with these options:

--dist=loadscope: tests will be grouped by module for test functions and by class for test methods, then each group will be sent to an available worker, guaranteeing that all tests in a group run in the same process. This can be useful if you have expensive module-level or class-level fixtures. Currently the groupings can’t be customized, with grouping by class takes priority over grouping by module. This feature was added in version 1.19.

--dist=loadfile: tests will be grouped by file name, and then will be sent to an available worker, guaranteeing that all tests in a group run in the same worker. This feature was added in version 1.21.