Is it possible to test that test-dependencies have not leaked into real code with python/pytest

Question

TL;DR: If I'm using pytest and some other test-only dependencies, is it possible to assert that none of these test-only dependencies have leaked into actual, non-test code?

In Java, when I run tests, the tests themselves and the code-under-test will run in different classloaders, with their own dependencies scoped there. So if the code-under-test inadvertently referenced testng, the tests will fail, even though testng is running the tests.

Can the same be achieved in Python? If my non-test code accidentally imports pytest, can I catch that and have the tests fail? I can't see how to achieve that.

Although setuptools and pip etc. make it relatively easy to keep install/run and dev/test dependencies separate, and even avoid polluting the current venv, they are still present when the tests run. This means that running python setup.py test for a module can pass, but python setup.py install followed by something as simple as importing the module can instafail.

Given: cat setup.py

from setuptools import setup, find_packages

setup(
    name="brettpy",
    version="0.0.1",
    packages=find_packages(),
    setup_requires=["pytest-runner",],
    tests_require=["pytest",],
)

cat setup.cfg

[aliases]
test=pytest

cat brettpy/__init__.py

import pytest

cat tests/test_brettpy.py

def test_import_brettpy():
    import brettpy
    del brettpy

def test_run_main():
    from brettpy import __main__ as main
    main.main()

... python setup.py test will pass, but python setup.py install && python -m brettpy will fail with:

ModuleNotFoundError: No module named 'pytest'

How do others ensure test-dependencies don't bleed into real code and cause missing-dependencies bugs when installed? Feels like a bug that the test-framework should be able to catch.

Answer 1

You can explicitly test for test imports by using Python's modulefinder. This is possibly not the prettiest of solutions:

from modulefinder import ModuleFinder

def test_imports():
    finder = ModuleFinder()
    finder.run_script("PATH_TO_MAIN_SCRIPT")
    tests_require = ["pytest",]  # you can also get this list straight from `setup.py`
    overlapping = [mod for mod in finder.modules.keys() for req in tests_require if req in mod]
    assert not overlapping

Or if you don't want to use a list comprehension:

    for mod_name in finder.modules.keys():
        for req in tests_require:
            assert req not in mod_name  # module could be pytest.something, _pytest, etc.