Cast and type env variables using file

Question

For all my projects, I load all env variables at the start and check that all the expected keys exist as described by an .env.example file following the dotenv-safe approach.

However, the env variables are strings, which have to be manually cast whenever they're used inside the Python code. This is annoying and error-prone. I'd like to use the information from the .env.example file to cast the env variables and get Python typing support in my IDE (VS Code). How do I do that?

env.example

PORT: int
SSL: boolean

Python Ideal Behavior

# Set the env in some way (doesn't matter)
import os
os.environment["SSL"] = "0"
os.environment["PORT"] = "99999"

env = type_env()
if not env["SSL"]: # <-- I'd like this to be cast to boolean and typed as a boolean
    print("Connecting w/o SSL!")
if 65535 < env["PORT"]:  # <-- I'd like this to be cast to int and typed as an int
    print("Invalid port!")

In this code example, what would the type_env() function look like assuming it only supported boolean, int, float, and str?

It's not too hard to do the casting as shown in e.g. https://stackoverflow.com/a/11781375/1452257, but it's unclear to me how to get it working with typing support.

Answer 1

I will suggest using pydantic.

From StackOverflow pydantic tag info

Pydantic is a library for data validation and settings management based on Python type hinting (PEP484) and variable annotations (PEP526). It allows for defining schemas in Python for complex structures.

let's assume that you have a file with your SSL and PORT envs:

with open('.env', 'w') as fp:
    fp.write('PORT=5000\nSSL=0')

then you can use:

from pydantic import BaseSettings

class Settings(BaseSettings):
    PORT : int
    SSL : bool
    class Config:
        env_file = '.env'

config = Settings()

print(type(config.SSL),  config.SSL)
print(type(config.PORT),  config.PORT)
# <class 'bool'> False
# <class 'int'> 5000

with your code:

env = Settings()

if not env.SSL:
    print("Connecting w/o SSL!")
if 65535 < env.PORT: 
    print("Invalid port!")

output:

Connecting w/o SSL!

Answer 2

The following solution offers both runtime casting to the desired types and type hinting help by the editor without the use of external dependencies.

Also check kederrac's answer for an awesome alternative using pydantic, which takes care of all of this for you.

---

Working directly with a non-Python dotenv file is going to be too hard, if not impossible. It's way easier to handle all the information in some Python data structure, as this lets the type checkers do their job without any modification.

I think the way to go is to use Python dataclasses. Note that although we specify types in the definition, they are only for the type checkers, not enforced at runtime. This is a problem for environment variables, as they are external string mapping information basically. To overcome this, we can force the casting in the __post_init__ method.

Implementation

First, for code organization reasons, we can create a Mixin with the type enforcing logic. Note that the bool case is special since its constructor will output True for any non-empty string, including "False". If there's some other non-builtin type you want to handle, you would need to add special handling for it, too (although I wouldn't suggest making this logic handle more than these simple types).

import dataclasses
from distutils.util import strtobool

class EnforcedDataclassMixin:

    def __post_init__(self):
        # Enforce types at runtime
        for field in dataclasses.fields(self):
            value = getattr(self, field.name)
            # Special case handling, since bool('False') is True
            if field.type == bool:
                value = strtobool(value)
            setattr(self, field.name, field.type(value))

This implementation can also be done with a decorator, see here.

Then, we can create the equivalent of a ".env.example" file like this:

import dataclasses

@dataclasses.dataclass
class EnvironmentVariables(EnforcedDataclassMixin):
    SSL: bool
    PORT: int
    DOMAIN: str

and for easy parsing from os.environ, we can create a function like

from typing import Mapping

def get_config_from_map(environment_map: Mapping) -> EnvironmentVariables:
    field_names = [field.name for field in dataclasses.fields(EnvironmentVariables)]
    # We need to ignore the extra keys in the environment,
    # otherwise the dataclass construction will fail.
    env_vars = {
        key: value for key, value in environment_map.items() if key in field_names
    }
    return EnvironmentVariables(**env_vars)

---

Usage

Finally, taking these things together, we can write in a settings file:

import os
from env_description import get_config_from_map


env_vars = get_config_from_map(os.environ)

if 65535 < env_vars.PORT:
    print("Invalid port!")

if not env_vars.SSL:
    print("Connecting w/o SSL!")

Static type checking works correctly in VS Code and mypy. If you assign PORT (which is an int) to a variable of type str, you will get an alert!

To pretend it's a dictionary, Python provides the asdict method in the dataclasses module.

env_vars_dict = dataclasses.asdict(env_vars)
if 65535 < env_vars_dict['PORT']:
    print("Invalid port!")

But sadly (as of the time of this answer) you lose static type checking support doing this. It seems to be work in progress for mypy.