erhosen-libs / pydjantic

Use Pydantic Settings in your Django application.

pydjantic

Use Pydantic Settings in your Django application.

Introduction

If you are tired of the mess in your Django Settings - I feel your pain:

• Ridiculously long settings.py file, with ASCII-art separation
• from common import * Python anti-pattern
• try: <import> except: ImportError Python anti-pattern
• base.py, production.py, local.py, domain.py - bunch of unrelated modules that override each other
• django-environ library, that do even worse...

On the other hand we have Pydantic Settings, which is de-facto standard for all non-django projects.

If you love Pydantic settings management approach, Pydjantic is a right tool for you.

Pydjantic allows you to define your settings in familiar way - just inherit from BaseSettings:

from typing import List

from pydantic import BaseSettings, Field
from pydantic.fields import Undefined
from pydjantic import to_django

class GeneralSettings(BaseSettings):
    SECRET_KEY: str = Field(default=Undefined, env='DJANGO_SECRET_KEY')
    DEBUG: bool = Field(default=False, env='DEBUG')
    INSTALLED_APPS: List[str] = [
        'django.contrib.admin',
        'django.contrib.auth',
    ]
    LANGUAGE_CODE: str = 'en-us'
    USE_TZ: bool = True


class StaticSettings(BaseSettings):
    STATIC_URL: str = '/static/'
    STATIC_ROOT: str = 'staticfiles'


class SentrySettings(BaseSettings):
    SENTRY_DSN: str = Field(default=Undefined, env='SENTRY_DSN')


class ProjectSettings(GeneralSettings, StaticSettings, SentrySettings):
    pass


to_django(ProjectSettings())

You can create as many classes/modules as you want, to achieve perfect settings' management. Divide your settings by domains, and then create final ProjectSettings class, that inherits from these domains.

Provide the instance of ProjectSettings to to_django function. That's all, your django settings will work as expected.

Installation

Install using pip install -U pydjantic or poetry add pydjantic.

Example

In the /demo directory you can find a working Django app with pydjantic settings.

Database configuration

Pydjantic comes with a special helper for managing DB configs - BaseDBConfig. See example below:

from pydantic import Field, PostgresDsn
from pydjantic import BaseDBConfig


class DatabaseConfig(BaseDBConfig):
    default: PostgresDsn = Field(default="postgres://user:password@hostname:5432/dbname", env="DATABASE_URL")

db_settings = DatabaseConfig()
assert db_settings.default == {
    'CONN_MAX_AGE': 0,
    'ENGINE': 'django.db.backends.postgresql_psycopg2',
    'HOST': 'hostname',
    'NAME': 'dbname',
    'PASSWORD': 'password',
    'PORT': 5432,
    'USER': 'user',
}

Also, you can define database configurations directly:

from pydantic import BaseSettings, Field

class PostgresDB(BaseSettings):
    ENGINE: str = 'django.db.backends.postgresql_psycopg2'
    HOST: str = Field(default='localhost', env='DATABASE_HOST')
    NAME: str = Field(default='dbname', env='DATABASE_NAME')
    PASSWORD: str = Field(default='password', env='DATABASE_PASSWORD')
    PORT: int = Field(default=5432, env='DATABASE_PORT')
    USER: str = Field(default='user', env='DATABASE_USER')
    OPTIONS: dict = Field(default={}, env='DATABASE_OPTIONS')
    CONN_MAX_AGE: int = Field(default=0, env='DATABASE_CONN_MAX_AGE')

class DatabaseConfig(BaseSettings):
    default = PostgresDB()

Or mix these approaches:

class DatabaseConfig(BaseDBConfig):
    default = Field(default="postgres://user:password@hostname:5432/dbname")
    replica = PostgresDB()

For more examples see tests.

Transformation from dsn to django format is done by dj-database-url library.

pydjantic's Issues

Pydantic serializer warnings, such as "Expected `multi-host-url` but got `dict` - serialized value may not be as expected"

Even when running tests:

tests/test_to_django.py::test_to_django_settings
  D:\NVIDIA\third-party\_python\pydjantic\.venv\Lib\site-packages\pydantic\main.py:308: UserWarning: Pydantic serializer warnings:
    Expected `multi-host-url` but got `dict` - serialized value may not be as expected
    return self.__pydantic_serializer__.to_python(

tests/test_db_config.py::test_dsn_and_exact_config
  D:\NVIDIA\third-party\_python\pydjantic\.venv\Lib\site-packages\pydantic\main.py:308: UserWarning: Pydantic serializer warnings:
    Expected `str` but got `dict` - serialized value may not be as expected
    return self.__pydantic_serializer__.to_python(

Seems to come from database configuration, e.g. (in test_dsn_and_exact_config):

class DatabaseConfig(BaseDBConfig):
    # declared as str, but actually deserialized as dict (Django database configuration)❓
    default: str = Field(default="postgres://user:password@hostname:5432/dbname")
    replica: PostgresDB = PostgresDB()

Applications still run, however:

• Users (like myself) might think that there is something wrong with their Django settings
• Application will fail to start under -Werror python interpreter flag

Type support

Hello, from what I can see, types are implemented, so, would it be possible to add the py.typed file for pylance, mypy and other type checker to work with this lib ?

bug in test_exact(): change BaseSettings to BaseDBConfig

pydjantic/tests/test_db_config.py

Line 79 in e7c4565

class DatabaseConfig(BaseSettings):

Right now we're testing Pydantic's serialization only with no Pydjantic code.
It should be DatabaseConfig(BaseDBConfig).

Support conn_max_age & ssl_require in BaseDBConfig

We should be able to set conn_max_age & ssl_require in BaseDBConfig.

feat: manage.py command to generate .env

Would be great to be able to bootstrap a .env based of the defined structure. Ideally interactive and non-interactive.

Improve failure output

I receive the following failure output:

SECRET_KEY
  field required (type=value_error.missing)

The SECRET_KET config looks as follows:

SECRET_KEY: str = Field(default=Undefined, env='DJANGO_SECRET_KEY')

Given I'm using environment variables to pass in the configuration I can not discern the issue from looking at the output, e.g. I don't know I have to provide a value in DJANGO_SECRET_KEY until looking into the code.

Would it be possible to provide more details, possibly the entire config for this field? I see there are title and description available, which could be helpful too:

SECRET_KEY: str = Field(default=Undefined, env='DJANGO_SECRET_KEY', title="Secret key", description="lorem ipsum...")

Possible solution:

SECRET_KEY
  Error: field required (type=value_error.missing)
  Title: Secret Key
  Description: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed suscipit, orci
      nec facilisis tristique, nibh leo pretium mi, sed consequat nibh magna sit amet elit. Maecenas
      tempus porta tellus eu feugiat. Phasellus eget vehicula libero. Donec eget augue
      pellentesque, condimentum elit sed, lobortis est. Fusce eu risus placerat, mollis dolor at,
      convallis mi.
  Env: DJANGO_SECRET_KEY

Support python3.8

Hi, would you mind changing minimal Requires-Python >=3.8?

SecretStr and SecretBytes values are not respected

Consider the following settings.py:

import os
from pydantic import BaseSettings, SecretStr
from pydjantic import to_django

class Settings(BaseSettings):
    MY_SECRET: SecretStr

# Define a secret in the env so that Pydantic validation passes
os.environ["MY_SECRET"] = "i am a secret"

settings = Settings()
print(settings.dict())   # ✅ Will NOT print the actual secret value, which is good

to_django(settings)

print(MY_SECRET)         # ❌ Prints "**********", which is incorrect

Here is the output of running the above snippet:

{'MY_SECRET': SecretStr('**********')}
**********

The second print should have yielded i am a secret instead. Otherwise there is no way of using the secret value in the rendered settings.