restart python (or reload modules) in py.test tests

Question

I have a (python3) package that has completely different behaviour depending on how it's init()ed (perhaps not the best design, but rewriting is not an option). The module can only be init()ed once, a second time gives an error. I want to test this package (both behaviours) using py.test.

Note: the nature of the package makes the two behaviours mutually exclusive, there is no possible reason to ever want both in a singular program.

I have serveral test_xxx.py modules in my test directory. Each module will init the package in the way in needs (using fixtures). Since py.test starts the python interpreter once, running all test-modules in one py.test run fails.

Monkey-patching the package to allow a second init() is not something I want to do, since there is internal caching etc that might result in unexplained behaviour.

• Is it possible to tell py.test to run each test module in a separate python process (thereby not being influenced by inits in another test-module)
• Is there a way to reliably reload a package (including all sub-dependencies, etc)?
• Is there another solution (I'm thinking of importing and then unimporting the package in a fixture, but this seems excessive)?

Answer 1

To reload a module, try using the reload() from library importlib

Example:

from importlib import reload
import some_lib
#do something
reload(some_lib) 

Also, launching each test in a new process is viable, but multiprocessed code is kind of painful to debug.

Example

import some_test
from multiprocessing import Manager, Process

#create new return value holder, in this case a list
manager = Manager()
return_value = manager.list()
#create new process
process =  Process(target=some_test.some_function, args=(arg, return_value))
#execute process
process.start()
#finish and return process
process.join()
#you can now use your return value as if it were a normal list, 
#as long as it was assigned in your subprocess

Answer 2

Once I had similar problem, quite bad design though..

@pytest.fixture()
def module_type1():
    mod = importlib.import_module('example')
    mod._init(10)
    yield mod
    del sys.modules['example']

@pytest.fixture()
def module_type2():
    mod = importlib.import_module('example')
    mod._init(20)
    yield mod
    del sys.modules['example']


def test1(module_type1)
    pass

def test2(module_type2)
    pass

The example/init.py had something like this

def _init(val):
    if 'sample' in globals():
        logger.info(f'example already imported, val{sample}' )
    else:
        globals()['sample'] = val
        logger.info(f'importing example with val : {val}')

output:

importing example with val : 10
importing example with val : 20

No clue as to how complex your package is, but if its just global variables, then this probably helps.