mypy spurious error: "module" has no attribute "XPath" with etree

Question

I'm trying to use mypy to do type checking in some code that uses the LXML library to parse XML.

On each line where I use etree.XPath, I get a spurious error from mypy. For example, the following trivial script

from lxml import etree    
NameXPath = etree.XPath("Name/text()")

generates the error

test.py:3: error: "module" has no attribute "XPath"

But the script runs fine and my XPath's work correctly at runtime.

I also tried #type:ignore on the import, which I thought might tell mypy not to type-check that library, but that did not suppress the errors.

from lxml import etree # type:ignore    
NameXPath = etree.XPath("Name/text()")

I did have some success at suppressing some of the errors by moving the calls to etree.XPath into a separate function which doesn't have any type annotations, but that seemed like a hack and forced me to arrange my code in awkward ways.

I would like to know if there's a way to completely suppress these spurious errors, or to possibly hint that the etree.XPath function does exist since it doesn't seem to be able to figure that out on its own.

To be clear, I don't actually care that mypy knows the correct types for structures coming out of the lxml library. I'm more concerned about putting the type info on my own classes that I'm shoving the parsed information into, so I want to have type-checked functions which use etree.XPath to do queries, find the data, and then shove them into type-annotated classes that are defined within my script.

mypy doesn't seem to have difficulty with other functions in etree, for example it's fine with my calls to etree.parse

I'm currently using mypy 0.4.4

Answer 1

It looks like this is a bug in typeshed, the community-contributed collection of type annotations for the stdlib and various third party libraries.

In particular, it looks as if the stubs for lxml are missing definitions for XPath entirely. This is probably an oversight -- I would try filing a bug on the issue tracker or try submitting a pull request containing a fix.

Once this is fixed, and mypy resyncs itself with the latest version of typeshed, you'll need to install mypy from the git repo for the time being (at least, until mypy 0.4.5 comes out sometime in October).

In the meantime, you can work around this by doing this:

from lxml.etree import XPath  # type: ignore
NameXPath = XPath("Name/text()")
# mypy considers NameXPath to have a type of Any

...or, if you'd prefer having a more specific definition of XPath, do this:

import typing

if typing.TYPE_CHECKING:
    # typing.TYPE_CHECKING is always False at runtime, so this
    # branch is parsed by typecheckers only
    class XPath:
        # Provide a method header stubs with signatures to fool
        # mypy into understanding what the interface for XPath is
else:
    # Actually executed at runtime
    from lxml.etree import XPath  # type: ignore

NameXPath = XPath("Name/text()")