In JSON created from a pydantic.BaseModel exclude Optional if not set

Question

I want to exclude all the Optional values that are not set when I create JSON. In this example:

from pydantic import BaseModel
from typing import Optional


class Foo(BaseModel):
    x: int
    y: int = 42
    z: Optional[int]


print(Foo(x=3).json())

I get {"x": 3, "y": 42, "z": null}. But I would like to exclude z. Not because its value is None, but because it is Optional and there was no keyword argument for z. In the two cases below I would like to have z in the JSON.

Foo(x=1, z=None)
Foo(x=1, z=77)

If there is any other solution to set z to optional in this sense, I would like to see it.

Answer 1

You could exclude only optional model fields that unset by making of union of model fields that are set and those that are not None.

Pydantic provides the following arguments for exporting method model.dict(...):

exclude_unset: whether fields which were not explicitly set when creating the model should be excluded from the returned dictionary; default False.

exclude_none: whether fields which are equal to None should be excluded from the returned dictionary; default False

To make union of two dicts we can use the expression a = {**b, **c} (values from c overwrites values from b). Note that since Python 3.9 it could be done just as a = b | c.

from pydantic import BaseModel
from typing import Optional
from pydantic.json import pydantic_encoder
import json


class Foo(BaseModel):
    x: int
    y: int = 42
    z: Optional[int]

def exclude_optional_dict(model: BaseModel):
    return {**model.dict(exclude_unset=True), **model.dict(exclude_none=True)}

def exclude_optional_json(model: BaseModel):
    return json.dumps(exclude_optional_dict(model), default=pydantic_encoder)
    


print(exclude_optional_json(Foo(x=3)))  # {"x": 3, "y": 42}
print(exclude_optional_json(Foo(x=3, z=None)))  # {"x": 3, "z": null, "y": 42}
print(exclude_optional_json(Foo(x=3, z=77)))  # {"x": 3, "z": 77, "y": 42}

Update

In order for the approach to work with nested models, we need to do a deep union( or merge) of two dictionaries, like so:

def union(source, destination):
    for key, value in source.items():
        if isinstance(value, dict):
            node = destination.setdefault(key, {})
            union(value, node)
        else:
            destination[key] = value

    return destination

def exclude_optional_dict(model: BaseModel):
    return union(model.dict(exclude_unset=True), model.dict(exclude_none=True))

class Foo(BaseModel):
    x: int
    y: int = 42
    z: Optional[int]

class Bar(BaseModel):
    a: int
    b: int = 52
    c: Optional[int]
    d: Foo


print(exclude_optional_json(Bar(a=4, d=Foo(x=3))))
print(exclude_optional_json(Bar(a=4, c=None, d=Foo(x=3, z=None))))
print(exclude_optional_json(Bar(a=4, c=78, d=Foo(x=3, z=77))))

{"a": 4, "b": 52, "d": {"x": 3, "y": 42}}
{"a": 4, "b": 52, "d": {"x": 3, "y": 42, "z": null}, "c": null}
{"a": 4, "b": 52, "c": 78, "d": {"x": 3, "y": 42, "z": 77}}