How To Get Pydantic To Discriminate On A Field Within List[Union[TypeA, TypeB]]?

Question

I am trying to use Pydantic to validate a POST request payload for a Rest API. A list of applicants can contain a primary and optional other applicant. So far, I have written the following Pydantic models listed below, to try and reflect this. The Rest API json payload is using a boolean field isPrimary to discriminate between a primary and other applicant.

from datetime import date
from pydantic import BaseModel, validator
from typing import List, Literal, Optional, Union


class PrimaryApplicant(BaseModel):
    isPrimary: Literal[True]
    dateOfBirth: Optional[date]


class OtherApplicant(BaseModel):
    isPrimary: Literal[False]
    dateOfBirth: date
    relationshipStatus: Literal["family", "friend", "other", "partner"]


class Application(BaseModel):
    applicants: List[Union[PrimaryApplicant, OtherApplicant]]

    @validator("applicants")
    def validate(
        cls,
        v: List[Union[PrimaryApplicant, OtherApplicant]]
    ) -> List[Union[PrimaryApplicant, OtherApplicant]]:

        list_count = len(v)
        primary_count = len(
            list(
                filter(lambda item: item.isPrimary, v)
            )
        )
        secondary_count = list_count - primary_count

        if primary_count > 1:
            raise ValueError("Only one primary applicant required")

        if secondary_count > 1:
            raise ValueError("Only one secondary applicant allowed")

        return v


def main() -> None:
    data_dict = {
        "applicants": [
            {
                "isPrimary": True
            },
            {
                "isPrimary": False,
                "dateOfBirth": date(1990, 1, 15),
                "relationshipStatus": "family"
            },
        ]
    }

    _ = Application(**data_dict)


if __name__ == "__main__":
    main()

With the example json payload listed above, when I try to remove some of the required mandatory fields from the OtherApplicant payload a ValidationError is correctly raised. For example, if I try to remove relationshipStatus or dateOfBirth field an error is raised. However, the isPrimary field is also reported by Pydantic to be invalid. Pydantic believes that this the isPrimary field should be True??? Example Pydantic validation output is listed below.

Why is Pydantic expecting that the isPrimary field should be True for an OtherApplicant list item in the json payload? Is it somehow associating the payload with PrimaryApplicant because of the use of Union? If so, how do I get Pydantic to use the isPrimary field to distinguish between primary and other applicants in the list payload?

Missing relationshipStatus field in list payload for OtherApplicant

pydantic.error_wrappers.ValidationError: 2 validation errors for Application
applicants -> 1 -> isPrimary
  unexpected value; permitted: True (type=value_error.const; given=False; permitted=(True,))
applicants -> 1 -> dateOfBirth
  field required (type=value_error.missing)

Missing dateOfBirth field in list payload for OtherApplicant

pydantic.error_wrappers.ValidationError: 2 validation errors for Application
applicants -> 1 -> isPrimary
  unexpected value; permitted: True (type=value_error.const; given=False; permitted=(True,))
applicants -> 1 -> relationshipStatus
  field required (type=value_error.missing)

Answer 1

Found the answer via also asking on Pydantic GitHub Repository

Pydantic 1.9 introduces the notion of discriminatory union.

After upgrading to Pydantic 1.9 and adding:

Applicant = Annotated[
    Union[PrimaryApplicant, OtherApplicant],
    Field(discriminator="isPrimary")]

It is now possible to have applicants: List[Applicant] field in my Application model. The isPrimary field is marked as being used to distinguish between a primary and other applicant.

The full code listing is therefore:

from datetime import date
from pydantic import BaseModel, Field, validator
from typing import List, Literal, Optional, Union
from typing_extensions import Annotated


class PrimaryApplicant(BaseModel):
    isPrimary: Literal[True]
    dateOfBirth: Optional[date]


class OtherApplicant(BaseModel):
    isPrimary: Literal[False]
    dateOfBirth: date
    relationshipStatus: Literal["family", "friend", "other", "partner"]


Applicant = Annotated[
    Union[PrimaryApplicant, OtherApplicant],
    Field(discriminator="isPrimary")]


class Application(BaseModel):
    applicants: List[Applicant]

    @validator("applicants")
    def validate(cls, v: List[Applicant]) -> List[Applicant]:

        list_count = len(v)
        primary_count = len(
            list(
                filter(lambda item: item.isPrimary, v)
            )
        )
        secondary_count = list_count - primary_count

        if primary_count > 1:
            raise ValueError("Only one primary applicant required")

        if secondary_count > 1:
            raise ValueError("Only one secondary applicant allowed")

        return v


def main() -> None:
    data_dict = {
        "applicants": [
            {
                "isPrimary": True
            },
            {
                "isPrimary": False,
                "relationshipStatus": "family"
            },
        ]
    }

    _ = Application(**data_dict)


if __name__ == "__main__":
    main()