Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why does mypy think library imports are missing?

Tags:

When I run mypy it complains that modules cannot be found:

sal@ahfang:~/workspace/ecs/cx-project-skeleton-repo/src/cx-example-function$ pipenv run python -m mypy . example_lambda.py:3: error: Cannot find module named 'aws_xray_sdk.core' 

But when trying to import that exact same module with the exact same Python interpreter, it seems that the module does exist and is importable.

python  Python 3.7.3 (default, Apr  3 2019, 05:39:12)  [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import aws_xray_sdk.core >>> 

Other than to force ignore the imports in the mypy.ini file, is there anything I should be doing to help mypy see importable modules that definitely do exist?

like image 378
Salim Fadhley Avatar asked Sep 04 '19 09:09

Salim Fadhley


People also ask

How do I ignore MYPY errors?

You can use a special comment # type: ignore[code, ...] to only ignore errors with a specific error code (or codes) on a particular line. This can be used even if you have not configured mypy to show error codes. Currently it's only possible to disable arbitrary error codes on individual lines using this comment.

Is installed but missing library stubs or PY typed marker?

Missing library stubs or py. typed marker. If you are getting a Skipping analyzing X: module is installed, but missing library stubs or py. typed marker , error, this means mypy was able to find the module you were importing, but no corresponding type hints.

What is MYPY?

“Mypy is an optional static type checker for Python that aims to combine the benefits of dynamic (or 'duck') typing and static typing. Mypy combines the expressive power and convenience of Python with a powerful type system and compile-time type checking.” A little background on the Mypy project.


2 Answers

So, here's the crux of the issue: mypy does not try type-checking every single module you've imported. Instead, it only attempts to type-check modules that have explicitly opted-in to the typing ecosystem.

Modules can opt-in to the typing ecosystem via two key mechanisms:

  1. Add type hints or stubs to their code, and include a file named py.typed within the package they distribute to PyPi (or any other package repository). The presence of this marker makes the package PEP-561-aware. The mypy docs also have more info about PEP-561-aware packages.
  2. Alternatively, add stubs to typeshed, the repository of type hints for the standard library and select 3rd party libraries.

The aws_xray_sdk package has done neither of these things, so will be ignored by mypy.


This is a bit unfortunate, so what can you do? The Missing imports section of the mypy docs has some detailed recommendations on what to do, but to summarize, you basically have three options which I'll list in order from least to most effort:

  1. Just silence the import by manually add # type: ignore comments to each import. You can also add the following section to your mypy config file to have this happen automatically:

    [mypy-aws_xray_sdk] ignore_missing_imports = True 

    Now, anything you import from this module will be treated as being of type Any.

  2. Search around and see if anybody has created a third party stubs package for your library: basically, an unofficial (or sometimes semi-official) PEP-561-aware package that only contains type hints. For example, for django, there's django-stubs, for SqlAlchemy, there's sqlalchemy-stubs.

  3. Create your own stubs for this library and point to them via the mypy_path option in your mypy config file:

    mypy_path = my_stubs/aws_xray_sdk, my_stubs/some_other_library 

    These stubs don't have to be complete, necessarily: you can get away with just adding annotations for the few things you're using. (And if they do end up becoming relatively complete, you perhaps look into contributing them back to the open-source community.)


Now finally, you may be wondering why mypy behaves this way?

Part of this is because it's not safe in the general case for mypy to just try finding and analyzing the module. Just blindly importing and using packages that are not type-hinting ready can sometimes result in odd type errors, or worse, can result in code incorrectly being marked as type-safe. That is, if you care about type-safety, it's better to be immediately notified that some package you're using has no type hints instead of mypy blindly inferring and smearing Anys all over your code.

Mypy could give a better error message here though, at least most of the time. IMO the fact that it doesn't is largely an oversight. There's some discussion about this in https://github.com/python/mypy/issues/4542.

like image 58
Michael0x2a Avatar answered Oct 05 '22 23:10

Michael0x2a


Sometimes, it could be as simple as adding an __init__.py file in your package.

I ran into the same issue, and realised that the package I was trying to import from, was missing the __init__.py file. Because of which mypy was not able to "see" the import.

like image 38
bool.dev Avatar answered Oct 05 '22 23:10

bool.dev