hacksoftware / django-enum-choices Goto Github PK

We should be able to support more Django versions.

Also, add tox to test them all

We need a Filter that works out of the box with django-filter

Consider that it needs to be aware of the choice_builder used by the corresponding model field

We encountered a pretty confusing bug today: When doing changes via the admin interface, sometimes some of our fields would contain a string instead of a proper enum value.

The same field on the same instance would be returned as an enum on a regular value, but as an empty string when the enum representation of an empty string is returned.

After some tinkering, we found that this piece of code might be the problem:

    def to_python(self, value):
        if value is None:
            return

        if value in self.empty_values:
            return ''

        return self._enum_from_input_value(value) or value

In our case value is an empty string so value in self.empty_values is True and the function returns an empty string instead of the actual enum representation.

We suggest changing the second if flow to

        if value in self.empty_values:
            value = ''

so that the empty representation is standardised and can be translated properly to an enum with _enum_from_input_value.

Do you agree with the suggested solution?

When setting blank=True in a model, the serializer implicitly inherits a serializer field with allow_blank=True. This, however, is being rejected by django_enum_choices.serializers.EnumChoiceField as an unknown argument in the __init__ call (since it only inherits from the regular serializers.Field).

The work around I applied in my code is to add allow_blank to dump_kwargs here

Do you agree this is the preferred solution? If so, I'm willing to submit a PR for it.

From an API perspective, I think it would be nicer to not need to specify blank=True at all (since you can derive that from the enum itself). However, it is needed as the admin menu of this model doesn't allow setting empty strings as the field on the admin menu form is marked as required=True otherwise. This might be a different discussion though.

Minimal (but untested) case to reproduce the issue:

import enum

from django.db import models
from django_enum_choices.choice_builders import attribute_value
from django_enum_choices.fields import EnumChoiceField


class Neo(models.Model):
    class PillChoice(enum.Enum):
        red = "Red"
        blue = "Blue"
        neither = "Chose neither pill"  # len(value) > max len(attr) of all attrs

    pill = EnumChoiceField(PillChoice, choice_builder=attribute_value)

We validate len(Neo.neither.value) while storing Neo.neither.name and setting max_length=len("neither")

Related to #38

https://semver.org/

• 57012fb#r34663285

• More examples / folder with examples
• Examples with auto()
• Describe corner cases (change the values without data migration)

Issue description

Changing an enum value and migrating the databse can lead to objects in the database containing the old enum value. No warning is shown during the migration process. Accessing such object results in ValidationError.

Steps to reproduce

Let's have the following enum:

class ExampleEnum(Enum):
    BAR = 'bar'

And a model that uses it:

class ExampleModel(Model):
    example_field = EnumChoiceField(ExampleEnum)

Create an instance(s) of the model:

ExampleModel.objects.create(example_field=ExampleEnum.BAR)

Change existing value in ExampleEnum:

class ExampleEnum(Enum):
    BAR = 'bar_updated'

Make a data migration and run it. Try to access the existing instance:

ExampleModel.objects.first()

The following error occurs:

ValidationError: ["Value bar not found in <enum 'ExampleEnum'>"]

Proposed solution

Document how to avoid this. Example:

At any given point of time all instances of a model that has EnumChoiceField must have a value that is currently present in the Enum class. If an enum value is changed/removed from the class ➡️ migration can be made and the database can be migrated successfully. No error or warning will be raised at this point, but future access to an instances containing the changed/removed enum value will result in a ValidationError:

ValidationError: ["Value {changed/removed-value-here} not found in <enum '{Enum-class-name-here}'>"]

If you need to change existing Enum value, or completely remove it, a data migration needs to be done first. All model instances that contain the enum value that's going to be changed, or removed, must be migrated to an enum value that exists in the Enum.

Note If the enum value is changed and the database migration has passed, you can still migrate the instances that contain the old enum value. You need to locate their ids and do:

YourModelName.objects.filter(id__in=[ids-of-instances-with-old-enum-value-here]).update(enum_field=<new-enum-value-here>)

• Resolve TODOs around code
• Write tests for with blank=True, null=True in a model
• Write tests for to_python

If we add an EnumChoiceField in list_filter, then the URLs being generated looke like this:

status__exact=Enum.SOMETHING

which fails with

AttributeError:
'str' object has no attribute 'value'

Hello 👋

In case you are still using django-enum-choices & you've not migrated to Django's 3.0 ( we have a migration guide - https://github.com/HackSoftware/django-enum-choices/wiki/Migrating-to-Django-3 ) - please, comment here.

We've closed all PRs & Issues, but if someone's still using this and cannot upgrade - we'll take a look.

Cheers!

In #37 we introduced a concept for end to end tests, in order to test migration generation & migration application.

The general cases we want to cover is:

1. Have a Django project in some state, with a model that uses EnumChoiceField
2. Change the Enum
3. Assert new migration is generated (with proper values)
4. Assert that after applying that migration, the database columns are successfully altered.

Currently, this is a standalone Python script - https://github.com/HackSoftware/django-enum-choices/blob/master/django_enum_choices/tests/e2e/tests.py - which runs only against Postgres with latest version of Django.

What we possibly want to achieve is:

1. Move that script into a py.test context
2. Somehow make it work with multiple databases (the major databases, supported by Django - sqlite, postgres for start)
3. Run it within tox context, to cover multiple versions of Python & Django

class StatusDEnum(Enum):
    WAITING = 'waiting'
    READY = 'ready'

status_d = EnumChoiceField(StatusDEnum, default=StatusDEnum.WAITING)

->

class StatusDEnum(Enum):
    WAITING = 'waiting'
    CANCELLED = 'cancelled'
    READY = 'ready'

->

python manage.py makemigrations
No changes detected

Scenario

DB is managed by someone else

MyTable.my_column contains values among ['Q1', 'Q01', 'Q2']

class MyEnum(Enum):
    Q1 = 'Q1'
    Q01 = 'Q1'
    Q2 = 'Q2'

Problem

django.core.exceptions.ValidationError: ["Value Q01 not found in <enum 'MyEnum'>"]

Solution

I haven't looked into a detailed solution, but I guess that it has to do with building the choices from enum_class.__members__ rather that iterating over enum_class

However I think it would break the choice_builder mechanic

If someone has a workaround, I'll take it.

I use this package for a project that i need have a default argument to pass one of enum choices to set as the first value of one that field in a model like other django classes and something like below e.g :

status = EnumChoiceField(StudentStatus, choice_builder=attribute_value, default=StudendStatus.Studying,verbose_name='student status')

and this feature is exactly like other base django models classes like CharField or IntegerField etc.

• Tests (and implementation) for serializer field's many behaviour
• Tests (and implementation) for serializer field's additional arguments (allow_blank, allow_null, required)

class StatusBEnum(Enum):
    WAITING = 'waiting'
    READY = 'ready'

status_B = EnumChoiceField(StatusBEnum, default=StatusBEnum.WAITING)

->

python manage.py makemigrations - OK
python manage.py migrate - OK
status_B varchar(7) - OK

->

class StatusBEnum(Enum):
    WAITING = 'waiting'
    CANCELLED = 'cancelled'
    READY = 'ready'

status_B = EnumChoiceField(StatusBEnum, default=StatusBEnum.WAITING)

->

python manage.py makemigrations - OK
python manage.py migrate - OK
status_B varchar(7) - WRONG - exptected varchar(9)

Scenario

from django.db import models

from enum import Enum

from django_enum_choices.fields import EnumChoiceField


class MyEnum(Enum):
    A = 'A'


class MyModel(models.Model):
    field = EnumChoiceField(MyEnum, blank=True, null=True)

With the following setup, we're trying to construct an enum choice field that can also be blank.

Django version: 2.2.12

Problem

The problem that we've hit is when we've tried to create an instance of MyModel with field = '', we receive the following error:

Value  not found in MyEnum

It says that an empty string ('') is an invalid value for the field, which is confusing as we're having blank=True already.

Potential fix

Tracing down the problem, we've reached its origin:

class EnumChoiceField(CharField):
    ...

    def to_enum_value(self, value):
        if value is None:  # <------------
            return

        for choice in self.enum_class:
            # Check if the value from the built choice matches the passed one
            if value_from_built_choice(self.choice_builder(choice)) == value:
                return choice

        raise ValidationError(
            _('Value {} not found in {}'.format(value, self.enum_class))
        )

When checking the value in the to_enum_value method, it is compared with is None. Which, when we have an empty string, evaluates to False. Thus the method continues with the interpretation and finally reaches the raise ValidationError statement.

When changing this line from:

if value is None:

to:

if not value:

It works as expected because it evaluates the value to falsy since it does not differentiate between None and '' (empty string).

Is this an expected behaviour?

Why custom EnumChoiceFieldException is used when builtin types are sufficient (TypeError and ValueError?

Example replacements:

• TypeErrors - utils.validate_built_choices,EnumChoiceField._get_choice_builder
• ValueErrors - EnumChoiceField.__init__

If you think that TypeError can be too much then ValueError can be enough in all cases but I still don't think that extra exception is worth it.

Probably I'm missing something (the way how you are using the library in your projects). If you think that my suggestions are meaningless just ignore them. Still if there are real reasons I will be glad if you share them with the audience.

If I have the following model:

class RecurringExpenseScheduleEnum(Enum):
    MONTHLY = 'monthly'
    WEEKLY = 'weekly'
    DAILY = 'daily'

    def __str__(self):
        return self.value


class RecurringExpense(BaseExpense, models.Model):
    """
    Serves as template from which expenses are generated on regular basis.
    """
    type = models.ForeignKey(
        ExpenseType,
        related_name='recurring_expenses',
        on_delete=models.CASCADE
    )

    schedule = EnumChoiceField(
        RecurringExpenseScheduleEnum,
        default=RecurringExpenseScheduleEnum.MONTHLY
    )

    def __str__(self):
        return f'Recurring {self.schedule} for {self.type} / {self.description}'

and the following admin:

@admin.register(RecurringExpense)
class RecurringExpenseAdmin(admin.ModelAdmin):
    exclude = ('created_at', )

    list_display = ('type', 'schedule', 'amount', 'description')
    list_select_related = ('type',)

Nothing's being displayed in Schedule.

Only when I change the admin to:

@admin.register(RecurringExpense)
class RecurringExpenseAdmin(admin.ModelAdmin):
    exclude = ('created_at', )

    list_display = ('type', 'get_schedule', 'amount', 'description')
    list_select_related = ('type',)

    def get_schedule(self, obj):
        return obj.schedule

    get_schedule.short_description = 'Schedule'

It's working

Django 3.0 includes enumeration types which do mostly the same thing as this library: https://adamj.eu/tech/2020/01/27/moving-to-django-3-field-choices-enumeration-types/

We have the following problem:

• If there's a default of an EnumChoiceField, when you open the Django admin for a specific object, that's always the selected value even if the actual value is something else.
• If there's no default, nothing's being selected in the dropdown