Best practices for generic functions in python

Question

I've read a lot about generic classes, and while those are cool, sometimes, I just need a generic function. Here's a small one I wrote:

def _matrix_map(self, mapper):
    """returns the matrix applying the mapper funcfunc"""
    return {key: mapper(value) for key, value in self._matrix.items()}

How is this supposed to be annotated for types. In a statically typed language with generic support, I'd write it something like this

private Dictionary<KeyType, ValueType> matrix;
private Dictionary<KeyType, T> matrix_map<T>(Func<ValueType, T>)

So, I thought I'd write it like this:

T = TypeVar('T')
def _matrix_map(self, mapper: Callable[[Tile], T]) -> Dict[Coordinate, T]:

I got it to pass with mypy like this, but pylint hates this.

On the first line:

C0103:Invalid class attribute name "T"

And on the second line:

E0602:Undefined variable 'T'

So I feel like I'm doing something wrong, and I could change the variable name (ttt? typ?), but that wouldn't solve the second problem. I can't be the first person to want (statically-typed) generic functions, but I can't find any good resources anywhere. Any recommendations?

Answer 1

I'm guessing you wrote code that looked roughly like this?

class Matrix:
    T = TypeVar('T')

    def _matrix_map(self, mapper: Callable[[Tile], T]) -> Dict[Coordinate, T]:
        # ...snip...

There's actually no issue with this at runtime/no issue for mypy, but it seems pylint isn't happy with you putting that "T" inside the scope of the class definition. What you can do to satisfy pylint is to instead do this:

T = TypeVar('T')

class Matrix:
    def _matrix_map(self, mapper: Callable[[Tile], T]) -> Dict[Coordinate, T]:
        # ...snip...

This is exactly equivalent to the former code snippet, and will also satisfy pylint.

(As a tangent, I'd argue that the fact that pylint isn't happy with the former is actually a bug. To be fair though, there aren't really any established style guides on best practices for type hints yet/putting a typevar inside of a nested scope is a somewhat uncommon use case, so I guess I can't really blame them for not accounting for this case.)

The only downside is that having that TypeVar on the outside does seem vaguely untidy/seems like it'll pollute the global namespace. You can partially mitigate this by renaming it to something like _T for example, to indicate it's private. You can also reuse this typevar whenever you need another generic class or function, which should also hopefully help keep the namespace pollution down.

For more information about using generics with Python type hints, see http://mypy.readthedocs.io/en/stable/generics.html