Dynamic list of valid str inputs for a Pydantic Field

Question

I need to pass in a list of valid inputs to a field in Pydantic, but I don't know what that list will include until runtime (logged-in users, date range for next 7 days, etc).

Passing in multiple values for this field is valid (selecting 2 users who have logged in is fine, but selecting 1 who hasn't is not).

Things I've tried look something like have revolved around Literal, List, and Union:

VALID = get_valid_inputs()

class ClassName(BaseModel):
    option_1: Literal[VALID] # Error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value
    option_2: List[VALID] # This does not throw an error, but also does not work the way I'm looking for. It requires a list with every value from VALID, whereas I'm looking for anywhere from 1, 2, or all of them.

I've done quite a lot of digging on this, but somehow haven't found anything, and I'm still relatively new to Pydantic. Apologies if it's a duplicate.

Answer 1

Dynamic type annotations are a contradiction in terms. It is theoretically impossible to perform static type checks on annotations that are only defined at runtime. This is also the reason, why things like Literal only support, well, literals and not arbitrary variables, unpacking or function calls.

For this reason, I would strongly recommend rethinking your design and if you really only know the types at runtime. If the situation is such that you do know the type of a field (e.g. str) in advance, but you want to restrict it to specific valid values, which you will only know about at runtime, you have a few options.

---

Dynamic definition of an Enum

Pydantic allows restricting choices via the standard library's enum.Enum type. And the EnumMeta provides a functional API, which allows you to define enum members dynamically.

Here is working demo:

from enum import Enum
from pydantic import BaseModel, ValidationError

def get_valid_inputs() -> tuple[str, ...]:
    return "foo", "bar", "baz"

...

Valid = Enum(  # type: ignore[misc]
    "Valid",
    ((value, value) for value in get_valid_inputs()),
    type=str,
)

class Model(BaseModel):
    single: Valid
    multiple: list[Valid] = []

def test() -> None:
    instance = Model(single="foo", multiple=["baz", "bar"])
    print(instance, "\n")
    try:
        Model(single="invalid")
    except ValidationError as e:
        print(e)

if __name__ == "__main__":
    test()

Output:

single=<Valid.foo: 'foo'> multiple=[<Valid.baz: 'baz'>, <Valid.bar: 'bar'>] 

1 validation error for Model
single
  value is not a valid enumeration member; permitted: 'foo', 'bar', 'baz' [...])

One drawback of this is that you'll have no static type checker support for the enum members because they are defined dynamically. E.g. your IDE will not give you useful hints/auto-suggestions for Valid members. In fact, mypy will complain about the way we define the enum because we did not provide a literal value as the second argument for Enum, which is why I put the type: ignore there.

---

Custom str subclass with validation

Pydantic also allows you to define custom data types that provide their own validation logic. You could therefore subclass str and set the valid choices on that class, while you define it. Then you could have a method for validation that simply checks if the provided value is among the valid choices. Here is an example for that approach:

from __future__ import annotations
from collections.abc import Callable, Iterator
from pydantic import BaseModel, ValidationError

...

class Valid(str):
    __choices__ = get_valid_inputs()

    @classmethod
    def __get_validators__(cls) -> Iterator[Callable[..., Valid]]:
        yield cls.validate

    @classmethod
    def __modify_schema__(cls, field_schema: dict[str, object]) -> None:
        field_schema.update(enum=list(cls.__choices__))

    @classmethod
    def validate(cls, v: str) -> Valid:
        if not isinstance(v, str):
            raise TypeError("string required")
        if v not in cls.__choices__:
            raise ValueError(f"'{v}' is invalid; choices: {cls.__choices__}")
        return cls(v)

class Model(BaseModel):
    single: Valid
    multiple: list[Valid] = []

...

if __name__ == "__main__":
    test()

Using the same get_valid_inputs and test functions as before, the output is equivalent:

single='foo' multiple=['baz', 'bar'] 

1 validation error for Model
single
  'invalid' is invalid; choices: ('foo', 'bar', 'baz') (type=value_error)

The __modify_schema__ method is not necessary, but illustrates how you can modify what the JSON schema of your model will look like, if you intend to use it with such a customized str type.

---

From the examples you mentioned briefly in your comment it still seems to me, that the valid options are actually to be considered static (i.e. not known strictly at runtime), just changing relatively frequently. If that is the case, you may want to consider actually hard-coding them. If you don't want to do that by hand every time they change, you can also look into datamodel generation programs.