Validate Python TypedDict at runtime

Question

I'm working in a Python 3.8+ Django/Rest-Framework environment enforcing types in new code but built on a lot of untyped legacy code and data. We are using TypedDicts extensively for ensuring that data we are generating passes to our TypeScript front-end with the proper data type.

MyPy/PyCharm/etc. does a great job of checking that our new code spits out data that conforms, but we want to test that the output of our many RestSerializers/ModelSerializers fits the TypeDict. If I have a serializer and typed dict like:

class PersonSerializer(ModelSerializer):
    class Meta:
        model = Person
        fields = ['first', 'last']

class PersonData(TypedDict):
    first: str
    last: str
    email: str

and then run code like:

person_dict: PersonData = PersonSerializer(Person.objects.first()).data

Static type checkers don't be able to figure out that person_dict is missing the required email key, because (by design of PEP-589) it is just a normal dict. But I can write something like:

annotations = PersonData.__annotations__
for k in annotations:
    assert k in person_dict  # or something more complex.
    assert isinstance(person_dict[k], annotations[k])

and it will find that email is missing from the data of the serializer. This is well and good in this case, where I don't have any changes introduced by from __future__ import annotations (not sure if this would break it), and all my type annotations are bare types. But if PersonData were defined like:

class PersonData(TypedDict):
    email: Optional[str]
    affiliations: Union[List[str], Dict[int, str]]

then isinstance is not good enough to check if the data passes (since "Subscripted generics cannot be used with class and instance checks").

What I'm wondering is if there already exists a callable function/method (in mypy or another checker) that would allow me to validate a TypedDict (or even a single variable, since I can iterate a dict myself) against an annotation and see if it validates?

I'm not concerned about speed, etc., since the point of this is to check all our data/methods/functions once and then remove the checks later once we're happy that our current data validates.

Answer 1

The simplest solution I found works using pydantic.

from typing import cast, TypedDict 
import pydantic


class SomeDict(TypedDict):
    val: int
    name: str

# this could be a valid/invalid declaration
obj: SomeDict = {
    'val': 12,
    'name': 'John',
}

# validate with pydantic
try:
    obj = cast(SomeDict, pydantic.create_model_from_typeddict(SomeDict)(**obj).dict())

except pydantic.ValidationError as exc: 
    print(f"ERROR: Invalid schema: {exc}")

EDIT: When type checking this, it currently returns an error, but works as expected. See here: https://github.com/samuelcolvin/pydantic/issues/3008

Answer 2

A little bit of a hack, but you can check two types using mypy command line -c options. Just wrap it in a python function:

import subprocess

def is_assignable(type_to, type_from) -> bool:
    """
    Returns true if `type_from` can be assigned to `type_to`,
    e. g. type_to := type_from

    Example:
    >>> is_assignable(bool, str) 
    False
    >>> from typing import *
    >>> is_assignable(Union[List[str], Dict[int, str]], List[str])
    True
    """
    code = "\n".join((
        f"import typing",
        f"type_to: {type_to}",
        f"type_from: {type_from}",
        f"type_to = type_from",
    ))
    return subprocess.call(("mypy", "-c", code)) == 0

Answer 3

You may want to have a look at https://pypi.org/project/strongtyping/. This may help.

In the docs you can find this example:

from typing import List, TypedDict

from strongtyping.strong_typing import match_class_typing


@match_class_typing
class SalesSummary(TypedDict):
    sales: int
    country: str
    product_codes: List[str]

# works like expected
SalesSummary({"sales": 10, "country": "Foo", "product_codes": ["1", "2", "3"]})

# will raise a TypeMisMatch
SalesSummary({"sales": "Foo", "country": 10, "product_codes": [1, 2, 3]})