Abstracting over type constructors in Python via type annotations

Question

I want to statically enforce that a method of a class returns a value wrapped in some abstract type, that I know nothing about:

E.g. given the abstract class

F = ???    

class ThingF(Generic[F]):
    @abstractmethod
    def action(self) -> F[Foo]:
        ...

I want to to be able to statically check that this is invalid:

class ThingI(ThingF[List]):
    def action(self) -> Foo:
        ...

because action does not return List[Foo].

However the above declaration for ThingF does not even run, because Generic expects its arguments to be type variables and I cannot find a way to make F a type variable "with a hole".

Both

F = TypeVar('F')

and

T = TypeVar('T')
F = Generic[T]

do not work, because either TypeVar is not subscriptable or Generic[~T] cannot be used as a type variable.

Basically what I want is a "higher kinded type variable", an abstraction of a type constructor, if you will. I.e. something that says "F can be any type that takes another type to produce a concrete type".

Is there any way to express this with Python's type annotations and have it statically checked with mypy?

Answer 1

Unfortunately, the type system (as described in PEP 484) does not support higher-kinded types -- there's some relevant discussion here: https://github.com/python/typing/issues/548.

It's possible that mypy and other type checking tools will gain support for them at some point in the future, but I wouldn't hold my breath. It would require some pretty complicated implementation work to pull off.

Answer 2

You can use Higher Kinded Types with dry-python/returns. We ship both primitives and a custom mypy plugin to make it work.

Here's an example with Mappable aka Functor:

from typing import Callable, TypeVar

from returns.interfaces.mappable import MappableN
from returns.primitives.hkt import Kinded, KindN, kinded

_FirstType = TypeVar('_FirstType')
_SecondType = TypeVar('_SecondType')
_ThirdType = TypeVar('_ThirdType')
_UpdatedType = TypeVar('_UpdatedType')

_MappableKind = TypeVar('_MappableKind', bound=MappableN)


@kinded
def map_(
    container: KindN[_MappableKind, _FirstType, _SecondType, _ThirdType],
    function: Callable[[_FirstType], _UpdatedType],
) -> KindN[_MappableKind, _UpdatedType, _SecondType, _ThirdType]:
    return container.map(function)

It will work for any Mappable, examples:

from returns.maybe import Maybe

def test(arg: float) -> int:
        ...

reveal_type(map_(Maybe.from_value(1.5), test))  # N: Revealed type is 'returns.maybe.Maybe[builtins.int]'

And:

from returns.result import Result

def test(arg: float) -> int:
    ...

x: Result[float, str]
reveal_type(map_(x, test))  # N: Revealed type is 'returns.result.Result[builtins.int, builtins.str]'

It surely has some limitations, like: it only works with a direct Kind subtypes and we need a separate alias of Kind1, Kind2, Kind3, etc. Because at the time mypy does not support variadic generics.

Source: https://github.com/dry-python/returns/blob/master/returns/primitives/hkt.py Plugin: https://github.com/dry-python/returns/blob/master/returns/contrib/mypy/_features/kind.py

Docs: https://returns.readthedocs.io/en/latest/pages/hkt.html

Announcement post: https://sobolevn.me/2020/10/higher-kinded-types-in-python