How do I correctly set MYPYPATH to pick up stubs for mypy?

Question

I can't for the life of me get MyPy to find stubs that aren't colocated with their source code. Here's the structure I have for my project:

trymypy/
|- stubs/
|  \- foo.pyi
|- __init__.py
|- usefoo.py
\- foo.py

# foo.py
def foofunc(x):
    return str(x)

# usefoo.py
from trymypy.foo import foofunc

print(foofunc(5) + 5)

# stubs/foo.pyi
def foofunc(x: int) -> str: ...

I have set the MYPYPATH environment variable to /full/path/to/trymypy/stubs, so that MyPy should look inside the stubs directory for my .pyi files.

This should not pass typechecking. MyPy should flag the error as such:

../trymypy/usefoo.py:3: error: Unsupported operand types for + ("str" and "int")

Instead, MyPy does not flag any errors, because it is not reading the stub file. If I move the stub file foo.pyi to the root project of the directory, colocated with foo.py, it correctly flags, which indicates to me that the MYPYPATH isn't getting picked up, or isn't being defined correctly.

I have also tried setting mypy_path in a config file mypy.ini:

[mypy]
python_version = 3.7
mypy_path = /full/path/to/trymypy/stubs

Other config options in mypy.ini do get picked up (e.g. python_version), so MyPy is seeing the file and reading it.

Totally stumped here. This is a (very) simple example that seems like it should be working as documented by MyPy. I've run out of variables to experiment on to get it working.

I am using Python 3.7 and working in a virtual environment that has only mypy installed.

Answer 1

Long story short: rather then having a separate 'stubs' directory, you probably want to move your foo.pyi file into the top-level trymypy folder, alongside foo.py.

---

Long story long, there are currently two problems with your setup.

The first problem is that the folder structure of your stubs need to mirror the way the underlying code is structured. So since you want to do from trymypy.foo import blah, you need to adjust your folder structure to look like this:

trymypy/
|- stubs/
|  |- trymypy/
|  |  |- __init__.pyi
|  \  \- foo.pyi
|- __init__.py
|- usefoo.py
\- foo.py

You should continue setting the mypypath to point to trymypy/stubs. You can use either an absolute or a relative path.

The bigger second problem is that your stubs may end up being shadowed and ignored depending on how exactly you're invoking mypy. For example, if you run mypy -p trymypy from outside of the trymypy folder, mypy will start by parsing every trymypy and the two submodules it directly contains (trymypy.__init__, trymypy.usefoo, and trymypy.foo).

And once trymypy.foo is loaded, mypy will not bother trying to re-load it again, which means it never bothers checking the stubs you specified.

But if you try type-checking individual files (e.g. mypy -m trymypy.usefoo, mypy -p trymypy.usefoo), mypy won't try loading everything inside trymypy, which means it can find the stubs using the typical import resolution rules.

You can confirm the behavior of all of this yourself by passing in the -v flag, which runs mypy in verbose mode and print out exactly what gets loaded. Be sure to delete the .mypy_cache directory before each run.

Note: I actually honestly have no idea whether this difference in behavior is intentional or a bug in mypy. The import rules are pretty nuanced.

---

The fix is thankfully straightforward though: just move your foo.pyi file into the top-level trymypy folder like so:

trymypy/
|- __init__.py
|- usefoo.py
|- foo.py
\- foo.pyi

Now, no matter what gets imported in what order, mypy will always find foo.py and foo.pyi at the same time, since both files are located in the same directory. And whenever a py and pyi file are located within the same directory, the pyi file always wins out (and the py file is ignored).

---

You may perhaps have two follow-up questions regarding this new folder structure:

1. Is there a way to type-check the contents of foo.py using the type hints present in foo.pyi?

   The answer is no, there currently is not a way. If you have foo.pyi present, the body of foo.py is essentially ignored altogether by mypy. There is somebody interested in adding support for this feature though, so you could perhaps subscribe to the linked Github issue for updates.

2. Create a separate "stubs" folder ended up not being useful here. So when is it useful?

   The answer is that it's primarily useful when you want to add stubs for third party libraries. I actually don't have a lot of experience with the "add stubs for your own code" workflow, but my understanding is that such stubs are usually "inlined" in the manner described above.