I have a function whose return type is sensitive to multiple arguments:
If a given predicate is strong enough to provide a type constraint,
the input values are similarly constrained (T | None -> T or T -> R where R <: T).
This is straightforward to type-hint if all types are known:
from typing import Any, Callable, Iterable, Iterator, TypeGuard, TypeVar, overload
T = TypeVar("T")
R = TypeVar("R")
@overload # 1
def select(pred: None, values: Iterable[T | None]) -> Iterator[T]: ...
@overload # 2
def select(pred: Callable[[T], TypeGuard[R]], values: Iterable[T]) -> Iterator[R]: ...
@overload # 3
def select(pred: Callable[[T], Any], values: Iterable[T]) -> Iterator[T]: ...
However, there is a problem if the argument types are not known:
If the predicate is Any, one would expect the least typing constraint and thus the same output type as the input values.
Yet, available type checkers do not agree1 with this:
MyPy looses the type almost completely to the overly generic Iterator[Any] and PyRight matches the None special case and provides the overconstraint Iterator[int].
How do I correctly type-hint an overload in which Any can match special cases?
Notably, reordering is not sufficient.
If I order overloads as 2-3-1 then Any does not provide an R and type checkers cannot fall back to T.
If I order overloads as 3-2-1 then (T) -> Any shadows (T) -> TypeGuard[R] completely.
1 Given a prelude of
from typing import Any, reveal_type
iitr: list[int | None] = [0, 1, 2, None, 3]
any_pred: Any = lambda val: not val # could be any unknown function
the type checkers I tested all fail in various ways when the unknown predicate is used:
| case | expected | MyPy | Pyright |
|---|---|---|---|
select(None, iitr) |
Iterator[int] |
Iterator[int] |
Iterator[int] |
select(bool, iitr) |
Iterator[int | None] |
Iterator[int | None] |
Iterator[int | None] |
select(any_pred, iitr) |
Iterator[int | None] |
Iterator[Any] |
Iterator[int] |
For Pyright, you can make use of Never to match a Any input.
from typing import Any, Callable, Iterable, Iterator, Never, TypeGuard, TypeVar, overload
T = TypeVar("T")
R = TypeVar("R")
@overload # Added
def select(pred: Never, values: Iterable[T]) -> Iterator[T]: ...
@overload # 1
def select(pred: None, values: Iterable[T | None]) -> Iterator[T]: ...
@overload # 2
def select(pred: Callable[[T], TypeGuard[R]], values: Iterable[T]) -> Iterator[R]: ...
@overload # 3
def select(pred: Callable[[T], Any], values: Iterable[T]) -> Iterator[T]: ...
For Mypy, I haven't thought of a solution yet, because there seems no way to indicate that R is a subtype of T, and Mypy adds all possible result types which would be unconstrained and therefore be Any.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With