Mypy reports an incompatible supertype error with overridden method

Question

Below is a simplified example of a problem I've encountered with mypy. The A.transform method takes an iterable of objects, transforms each one (defined in the subclass B, and potentially other subclasses) and returns an iterable of transformed objects.

from typing import Iterable, TypeVar

T = TypeVar('T')

class A:
    def transform(self, x: Iterable[T]) -> Iterable[T]:
        raise NotImplementedError()

class B(A):
    def transform(self, x: Iterable[str]) -> Iterable[str]:
        return [x.upper() for x in x]

However mypy says:

error: Argument 1 of "transform" incompatible with supertype "A"
error: Return type of "transform" incompatible with supertype "A"

If I remove [T] from A.transform(), then the error goes away. But that seems like the wrong solution.

After reading about covariance and contravariance, I thought that setting T = TypeVar('T', covariant=True) might be a solution, but this produces the same error.

How can I fix this? I have considered binning the design altogether and replacing the A class with a higher order function.

Answer 1

Making T covariant or contravariant isn't really going to help you in this case. Suppose that the code you had in your question was allowed by mypy, and suppose a user wrote the following snippet of code:

def uses_a_or_subclass(foo: A) -> None:
    # This is perfectly typesafe (though it'll crash at runtime)
    print(a.transform(3))

# Uh-oh! B.transform expects a str, so we just broke typesafety!
uses_a_or_subclass(B())  

The golden rule to remember is that when you need to overwrite or redefine a function (when subclassing, like you're doing, for example), that functions are contravariant in parameters, and covariant in their return type. This means that when you're redefining a function, it's legal to make the parameters more broad/a superclass of the original parameter type, but not a subtype.

One possible fix is to make your entire class generic with respect to T. Then, instead of subclassing A (which is now equivalent to subclassing A[Any] and is probably not what you want if you'd like to stay perfectly typesafe), you'd subclass A[str].

Now, your code is perfectly typesafe, and your redefined function respects function variance:

from typing import Iterable, TypeVar, Generic

T = TypeVar('T')

class A(Generic[T]):
    def transform(self, x: Iterable[T]) -> Iterable[T]:
        raise NotImplementedError()

class B(A[str]):
    def transform(self, x: Iterable[str]) -> Iterable[str]:
        return [x.upper() for x in x]

Now, our uses_a_or_subclass function from up above should be rewritten to either be generic, or to accept specifically classes that subtype A[str]. Either way works, depending on what you're trying to do.