"extra fields not permitted" when subclassing model with private fields

Question

I have two classes:

from pydantic import BaseModel, Extra

class Foo(BaseModel):
    a: str
    class Config:
        extra = Extra.forbid

class Bar(Foo):
    _secret: str

When I try initializing Bar:

Bar(a='a', _secret='b')

I get the following error:

pydantic.error_wrappers.ValidationError: 1 validation error for Bar
secret
  extra fields not permitted (type=value_error.extra)

• This works fine if Bar is just subclassed off BaseModel, however I need it subclassed off of Foo for a variety of other reasons

• This works fine if _secret is just secret, but its really something I'd prefer to have as a hidden attribute

• This works fine if Foo does not have Extra.forbid, but its in a library that I'm importing and cannot change.

I'm not sure what else to try, any suggestions?

Answer 1

By default, Pydantic basically disregards underscored attributes completely. They are not added as fields to the model. In this case, arbitrary non-field attributes are also forbidden via the extra setting. If you now try to pass _secret="b" to the constructor, you get an error message because _secret is not a field and you tried to pass an "extra" value.

To solve this, you can override the __init__ method and set your _secret attribute there, but take care to call the parent __init__ with all other keyword arguments. In addition, you will need to declare _secret to be a private attribute, either by assigning PrivateAttr() to it or by configuring your model to interpret all underscored (non-class-)attributes as private.

This works:

from typing import Any
from pydantic import BaseModel, Extra


class Foo(BaseModel):
    a: str

    class Config:
        extra = Extra.forbid


class Bar(Foo):
    _secret: str

    def __init__(self, _secret: str, **kwargs: Any) -> None:
        self._secret = _secret
        super().__init__(**kwargs)

    class Config:
        underscore_attrs_are_private = True


bar = Bar(a="a", _secret="b")
print(bar)          # a='a'
print(bar._secret)  # b

The only downside I can see is that this seems to break some of the functionality of the Pydantic PyCharm plugin for suggesting __init__ parameters. But that is not too dramatic IMO.

If you want to avoid overriding the __init__ method, you will have to write some setter method for _secret instead, but it still must be declared as a private attribute.

Answer 2

If you are using pydantic 2 with pydantic-settings and BaseSettings instead of BaseModel, then set the config value of extra to allow or ignore. This is the new way of ignoring the extra configs in pydantic 2.

The error message I was getting:

Extra inputs are not permitted [type=extra_forbidden, input_value='Collections', input_type=str] For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden

My fix for the error:

from pydantic_settings import BaseSettings, SettingsConfigDict
    
class AppSettings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=".env",
        env_prefix="APP_",
        extra="ignore"
    )

    ENVIRONMENT: str
    PORT: int
    ECHO_SQL: bool
    DEBUG: bool