pytest fails with ModuleNotFoundError and name of unused plugin

Question

I maintain an open source library, xarray, which runs integration tests on Travis-CI using pytest. We install the scientific Python using stack conda.

Earlier today, our two out of our five test builds (Python 3.5 and 3.6, but not Python 2.7 or 3.4) started failing for no apparent reason. pytest itself fails, with an mystifying trackback:

$ py.test xarray --cov=xarray --cov-config ci/.coveragerc --cov-report term-missing --verbose $EXTRA_FLAGS
Traceback (most recent call last):
  File "/home/travis/miniconda/envs/test_env/bin/py.test", line 6, in <module>
    sys.exit(py.test.main())
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/config.py", line 49, in main
    config = _prepareconfig(args, plugins)
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/config.py", line 168, in _prepareconfig
    pluginmanager=pluginmanager, args=args)
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 613, in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 250, in _wrapped_call
    wrap_controller.send(call_outcome)
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/helpconfig.py", line 68, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 279, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 265, in __init__
    self.result = func()
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/config.py", line 957, in pytest_cmdline_parse
    self.parse(args)
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/config.py", line 1121, in parse
    self._preparse(args, addopts=addopts)
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/config.py", line 1084, in _preparse
    self.pluginmanager.load_setuptools_entrypoints('pytest11')
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 510, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2409, in load
    return self.resolve()
  File "/home/travis/miniconda/envs/test_env/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2415, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'hypothesis.extra.pytestplugin'

A few things are surprising things to me about this:

1. We don't use hypothesis or install hypothesis.extra.pytestplugin. I have no idea where this came from.
2. There's nothing obvious that changed to trigger this error. It first appeared after a one line doc change. The installed Python libraries, including the specific binary builds installed by conda, appear to be exactly the same. (You can diff the build outputs yourself if you like, though.)

If you have any clues or guesses about what's going on that, your help would be appreciated! I don't even know where in the stack to file a bug.

Answer 1

I dug into this a bit:

• Conda forge gives us hypothesis: 3.33.0-py36_0 and pytest is loading it as a plugin
• Pinning the conda version of hypothesis to 3.23 corrects the problem.
• Uninstalling hypothesis solves the problem
• Pip installing hypothesis solves the problem

So, I think we can conclude something went wrong with the conda-forge build of hypothesis.

There is an open issue on the conda-forge page for this:

https://github.com/conda-forge/hypothesis-feedstock/issues/16