I don't understand the reason for this yet, but djangorestframework-dataclasses currently fails when using forward references with PEP 585 collection syntax, for example list["SomeClass"].

It works, however, when using typing.List["SomeClass"] or list[SomeClass] (without a forward reference).

I wrote this test case to reproduce the issue:

@dataclasses.dataclass
class Simple:
    value: str


class IssuesTest(TestCase):
    # Issue #51: Type forward references do not work with PEP 585 collections
    def test_forward_reference_list(self):
        @dataclasses.dataclass
        class WithForwardReference:
            """Use quoted type hints (e.g. forward references from PEP 484)"""
            child: 'Simple'
            # children: typing.List['Simple']   <-- works when uncommenting this
            children: list['Simple']

        serializer = DataclassSerializer(
            dataclass=WithForwardReference, data={'child': {'value': 'test1'}, 'children': [{'value': 'test2'}]}
        )
        serializer.is_valid(raise_exception=True)
        data: WithForwardReference = serializer.validated_data

        self.assertEqual('test1', data.child.value)
        self.assertEqual('test2', data.children[0].value)

The result is:

NotImplementedError: Automatic serializer field deduction not supported for field 'children' on 'WithForwardReference' of type 'list['Simple']' (during search for field of type 'Simple').

If you have something like:

@dataclass
class Person():
    name: str


@dataclass
class Company():
    people: List[Person]


class PersonSerializer(DataclassSerializer):
    class Meta:
        dataclass = Person

class CompanySerializer(DataclassSerializer):
    class Meta:
        dataclass = Company

If I use get_schema_view to generate an openapi schema, the result for a view using CompanySerializer is this:

              properties:
                people:
                  type: array
                  items:
                    type: object # <---- no typing for the child elements!
                  writeOnly: true
              required:
              - people

However, if I use PeopleSerializer directly, it generates a schema for all fields as expected.
I'm not sure if this is a problem with DRF dataclass serializer or somewhere else, because get_fields() on the serializer is definitely returning all fields as expected, and not sure how to debug this. Any ideas on why this is happening?

It looks like DataclassSerializer can't handle properties automatically. It would be great to make this possible:

@dataclass
class PricingInfo:
    original_price_incl_tax: Decimal

    @property
    def original_price(self) -> Decimal:
        """Backwards compatibility"""
        return self.original_price_incl_tax

class PricingInfoSerializer(DataclassSerializer):
    class Meta:
        dataclass = PricingInfo
        fields = (
            "original_price",
        )

The field 'original_price' was included on serializer PricingInfoSerializer in the `fields` option, but does not match any dataclass field.

Hi there!

I have a situation here, that I think might be a bug, or maybe I just haven't found a way to solve it since I'm still new with the package.

Example:

from dataclasses import dataclass, field

@dataclass
class A:
    foo: str
    bar: str = field(init=False)

from rest_framework_dataclasses.serializers import DataclassSerializer

class ASerializer(DataclassSerializer):
    class Meta:
        dataclass = A

The previous code (adapted and simplified from my real code), raises the exception when trying to validate:

TypeError: A.__init__() got an unexpected keyword argument 'bar'

Proposal:
In the following line of code, where the dataclass is being instantiated, it might be a good idea to exclude from the empty_values, those fields with init=False.

I'm trying to build an API for a large body of synthetic data. By synthetic I mean it's put together from a pile of models.

The approach I'm trying is:
0: Define dataclasses that describe the synthetic structure.
1: Assemble instances of those data classes with the calculated values.
2: Throw the whole lot at a DataclassSerializer.

To give you a sense of the size of what I'm messing with I have 12 dataclasses and there's about 40 points where they nest inside each other. Pretty formatted minimal output for one detail view is over 300 lines of JSON.

On the get side this is all going silky smooth. Thank you, that alone has saved me a ton of hassle.

Now I need to allow patching into this structure, the client is allowed to send any sub portion of the 300 lines as a patch, which means any subset of the underlying fields and subsequently leaving out large amounts of stuff that is not optional at the dataclass level.

What I've done that is working but a lil clunky is:
1: Assemble the dataclasses for the current state (same as 1 above).
2: Push that through the DataclassSerializer to get JSON (same as 2 above).
3: Apply the incoming JSON as a patch to the JSON from 2.
4: Feed that into the data argument of the DataclassSerializer. MySerializer(data=<JSON from 3>)
5: Get the updated dataclasses from the serializers validated data.
6: Compare the dataclasses from 1 with the dataclasses from 5 and action any differences.
7: If there were no errors return the serializer from 4 to the API.

A: Is there a better way I should be doing this?
Particularly it seems like I should be able to skip 2 and 3 and pass the JSON patch into the serializer at 4 along with the dataclasses from one and partial=True and get the same result, but doing that results in lots of missing field errors on the nested dataclasses.

B: Also, at 4 if I pass the dataclasses from 1 as instance= then it seems to ignore the values in data, but if I pass it as initial= then it works as expected, which is odd. Although with the setup above there's no particular need pass it at all.

C: I'm not sure how to define readonly fields through this mess, it's not critical as I only check the writable stuff at 6, but it'd be nice. I don't want to build out serializers for it all as that removes a lot of the value I get from DataclassSerializer.
The writable stuff is only in a couple of dataclasses, is there a way I could define serializers for them and then tell the top level serializer if you find a dataclass X anywhere in the structure, use the X serializer?

Hi!

I found a problem with serializers like:

class MySerializer(...):
    my_nested = MyDataclassSerializer(allow_null=True, required=False)

The fix:

def create(self, validated_data):
    if validated_data is None:
        return None

    dataclass_type = self.get_dataclass_type()
    return dataclass_type(**self.instantiate_data_dictionaries(validated_data))

Hello,

djangorestframework-dataclasses is helping me document my API with drf-spectacular.
Thanks for that.

I noticed a inherited dataclass serializer is different than one created by a function.
An example:

@dataclass
class Monty:
    python: bool

MontySerializer = DataclassSerializer(dataclass=Monty)
class MontyClassSerializer(DataclassSerializer):
    class Meta:
        dataclass = Monty

monty = Monty(True)
MontyClassSerializer(instance = monty) # works
MontySerializer(instance=monty) # TypeError: 'DataclassSerializer' object is not callable

My expectation is that both, class and function would have the same result.
It really doubles my code when it needs to be created as an actual class.
Is there a way around this?

If there are multiple nested different data classes in one data class, the swagger doc will only contain one nested data class type.

We can solve the problem by specifying the nested data class serializer manually. The annotation is here, https://drf-yasg.readthedocs.io/en/stable/custom_spec.html#the-swagger-auto-schema-decorator

According to https://www.django-rest-framework.org/api-guide/fields/#using-source, the serialisers should allow for use of source="*".

The issue can be reproduced in the following way. Copy the following snippet on the bottom of tests/test_functional.py:

class PetEmbeddedDefaultSerializer(DataclassSerializer):
    class PetInformationSerializer(DataclassSerializer):
        animal = fields.CharField()

    class Meta:
        dataclass = Pet
        fields = ['name', 'information']

    name = fields.CharField()
    information = PetInformationSerializer(source="*")


class SourceStartTest(TestCase):
    DATA = {'name': 'Milo', 'information': {'animal': 'cat'}}
    INSTANCE = Pet(name='Milo', animal='cat')

    def test_default_serialization(self):
        serializer = PetEmbeddedDefaultSerializer(instance=self.INSTANCE)
        self.assertDictEqual(serializer.data, self.DATA)

    def test_default_deserialization(self):
        serializer = PetEmbeddedDefaultSerializer(data=self.DATA)
        serializer.is_valid(raise_exception=True)
        self.assertEqual(serializer.instance, self.INSTANCE)

I am currently using https://github.com/axnsan12/drf-yasg which does a lot of introspection on serializers. Using a dataclass with an enum causes it to fail due to it expecting to be able to pass a string to the ChoiceField to_representation method, but this library expects the passed value to only be an enum.

Snippet to show how drf_yasg is trying to call the to_representation method:

if isinstance(field, serializers.ChoiceField):
    enum_type = openapi.TYPE_STRING
    enum_values = []
    for choice in field.choices.keys():
        if isinstance(field, serializers.MultipleChoiceField):
            choice = field_value_to_representation(field, [choice])[0]
        else:
            choice = field_value_to_representation(field, choice)

        enum_values.append(choice)

Obviously this introspection is not the normal use case, but it would be sweet if it could could be compatible with ChoiceFields method. Since there is the by_name feature, I am not sure if there is a simple solution other than perhaps type checking?

ChoiceFields to_representation for reference:

def to_representation(self, value):
    if value in ('', None):
        return value
    return self.choice_strings_to_values.get(str(value), value)
...
self.choice_strings_to_values = {
    str(key): key for key in self.choices
}

If you pass a QueryDict to a serializer (such as request.GET), the bool somehow loses it's default value.

Demonstration:

from dataclasses import dataclass
from rest_framework_dataclasses.serializers import DataclassSerializer


@dataclass
class MyClass:
    my_bool: bool = True


class MySerializer(DataclassSerializer):
    class Meta:
        dataclass = MyClass


from django.http import QueryDict
from django.conf import settings

settings.configure()
s = MySerializer(data=QueryDict(encoding='utf8'))
assert s.is_valid()
assert s.data['my_bool'] is True, "this should be true but is false!"

We should probably revisit the fixtures to reduce the three-level nesting and feature duplication in them, and move the functional tests to a separate test unit.

Probably I am not using dataclass properly as this looks like a simple issue, here is my code

@dataclasses.dataclass
class OrderItem(TypedJsonMixin):
    product_id: typing.Union[uuid.UUID, str]
    unit_price: float
    quantity: int
    product_type: str
    discount: float
    status: typing.Optional[str] = ""
    description: typing.Optional[str] = None
    message: typing.Optional[str] = None  # Order status message.


class ItemSerializer(DataclassSerializer):
    event = serializers.SerializerMethodField()
    product_id = serializers.UUIDField(required=True)

    class Meta:
        dataclass = OrderItem
        # fields = "__all__"
        read_only_fields = ("description", "message")
        extra_kwargs = {
            # "product_id": {"max_length": 36, "required": True},
            "product_type": {"required": True},
            "status": {"default": "in-cart"},
        }

Now while creating an instance of ItemSerializer, I am not passing description and message.

data = {
	      "product_id": "a503e2e0-ce0e-4556-bb58-08d4f99f199e",
	      "product_type": "public-event",
	      "unit_price": 500,
	      "quantity": 1,
	      "discount": 10
	    }
s = ItemSerializer(data=data)
s.is_valid()

I am getting the following error, as empty_values is rest_framework.fields.empty

Error

serializer.is_valid(raise_exception=True)
  File "/Users/arun/.local/share/virtualenvs/djZipDate-ChKVyfQ3/lib/python3.8/site-packages/rest_framework/serializers.py", line 234, in is_valid
    self._validated_data = self.run_validation(self.initial_data)
  File "/Users/arun/.local/share/virtualenvs/djZipDate-ChKVyfQ3/lib/python3.8/site-packages/rest_framework/serializers.py", line 433, in run_validation
    value = self.to_internal_value(data)
  File "/Users/arun/.local/share/virtualenvs/djZipDate-ChKVyfQ3/lib/python3.8/site-packages/rest_framework/serializers.py", line 490, in to_internal_value
    validated_value = field.run_validation(primitive_value)
  File "/Users/arun/.local/share/virtualenvs/djZipDate-ChKVyfQ3/lib/python3.8/site-packages/rest_framework/serializers.py", line 621, in run_validation
    value = self.to_internal_value(data)
  File "/Users/arun/.local/share/virtualenvs/djZipDate-ChKVyfQ3/lib/python3.8/site-packages/rest_framework/serializers.py", line 657, in to_internal_value
    validated = self.child.run_validation(item)
  File "/Users/arun/.local/share/virtualenvs/djZipDate-ChKVyfQ3/lib/python3.8/site-packages/rest_framework/serializers.py", line 433, in run_validation
    value = self.to_internal_value(data)
  File "/Users/arun/.local/share/virtualenvs/djZipDate-ChKVyfQ3/lib/python3.8/site-packages/rest_framework_dataclasses/serializers.py", line 531, in to_internal_value
    instance = dataclass_type(**native_values, **empty_values)
  File "<string>", line 11, in __init__
  File "/Users/arun/.local/share/virtualenvs/djZipDate-ChKVyfQ3/lib/python3.8/site-packages/typed_json_dataclass/typed_json_dataclass.py", line 67, in __post_init__
    raise TypeError((f'{class_name}.{field_name} was '
TypeError: OrderItem.description was defined to be any of: (<class 'str'>, <class 'NoneType'>) but was found to be <class 'type'> instead

I read some existing issues about optional fields, but do not see a solution.

Currently, if extra data fields are passed into a DataclassSerializer instance that are not modeled by the underlying @dataclass, they are silently ignored. This could hide subtle bugs in clients that think they are sending a particular field, but in fact the field is being ignored and they are not made aware of it.

For example:

from dataclasses import dataclass

from rest_framework_dataclasses.serializers import DataclassSerializer


@dataclass
class Person:
    name: str
    height: float


class PersonSerializer(DataclassSerializer[Person]):
    class Meta:
        dataclass = Person


p = PersonSerializer(
    data={
        "name": "Bruce",
        "height": 6.2,
        "im_batman": True,  # Extra, invalid field
    }
)
p.is_valid(raise_exception=True)  # Be able to configure the DataclassSerializer to fail validation here

A partial implementation (does not work with nested DataclassSerializers):

class StrictDataclassSerializer(DataclassSerializer[T], Generic[T]):
    def validate(self, attrs: dict[str, Any]) -> dict[str, Any]:
        # Need to check for initial_data as nested serializers will not have this field.
        if hasattr(self, "initial_data"):
            unknown_keys = set(self.initial_data.keys()) - set(fields.keys())
            if unknown_keys:
                raise ValidationError(f"Unknown keys found: {unknown_keys}")
        return attrs

A feature like this could be enabled in the Meta:

class PersonSerializer(DataclassSerializer[Person]):
    class Meta:
        dataclass = Person
        strict = True

Hey,
I am using this great lib and found problems with drf-spectacular. These are my issues:

If I have nested dataclass like:

@dataclass
class A():
   foo: str

class B():
   a: A
   
# serializer 
class BSerializer(DataclassSerializer):
  class Meta:
    dataclass = B

# the same results with  DataclassSerializer(B)
 

Then final docs do not recognise fields of A saying: object (Dataclass)

When I try:

# serializer 
class BSerializer(DataclassSerializer):
  a = DataclassSerializer(A)
  class Meta:
    dataclass = B

It does not give any effect, I still have a: object (Dataclass)

In order to have it working I need to define for every nested dataclass its own empty serializer like:

class ASerializer(DataclassSerializer):
   class Meta:
     dataclass = A
     
class BSerializer(DataclassSerializer):
  a = ASerializer()
  class Meta:
    dataclass = B

What gives a lot of such proxy code (imagine few such dataclasses inside)
Also docs print all the time comment from the DataclassSerializer, so I see :

object (Dataclass)A DataclassSerializer is just a regular Serializer, except that:A set of default fields are automatically populated.A set of default validators are automatically populated.Default .create() and .update() implementations are provided.The process of automatically determining a set of serializer fields based on the dataclass fields is slightly complex, but you almost certainly don't need to dig into the implementation.If the DataclassSerializer class doesn't generate the set of fields that you need you should either declare the extra/differing fields explicitly on the serializer class, or simply use a Serializer class.

For every field using DataclassSerializer.

In summary using these two things together looks like quite a struggle.

I have a dataclass that contains a frozenset field. When I use save() on my serializer the created object contains a list though:

@dataclass
class Foo:
    bar: frozenset[str] | None = None


class FooSerializer(DataclassSerializer):
    class Meta:
        dataclass = Foo

class FooView(APIView):
    def post(self, request, format=None):
        serializer = FooSerializer(data=request.data)
        if serializer.is_valid():
            foo = serializer.save()
            # type of foo.bar is list

I already had a look at serializer_field_mapping and tried to add frozenset to it, but found no way to make it work.

What can I do to make the serializer automatically turn the list into a frozenset?

Why lock on a dataclass? Why not just any class?

Hello, I'm really impressed with the work you've done with this library, but I have a small issue working with it.

I have the next dataclasses declared:

from dataclasses import dataclass, field


@dataclass
class SocialSecurityNumber:
    number: str

    @property
    def hidden(self) -> str:
        return f'SSN <#{self.number}>'

    def __str__(self) -> str:
        return self.hidden


@dataclass
class Profile:
    email: str
    social_security_number: SocialSecurityNumber


@dataclass
class Group:
    name: str


@dataclass
class User:
    id: int
    username: str
    profile: Profile
    groups: list[Group]

Also, I have implemented my own serializer field to hide the social security number value (notice the constructor):

from rest_framework.fields import Field


class SocialSecurityNumberField(Field):
    def __init__(self, *args, **kwargs):
        # kwargs.pop('dataclass', None)
        # kwargs.pop('many', None)
        super().__init__(*args, **kwargs)

    def to_representation(self, value):
        print('Called to_representation')
        return value.hidden

And I have my own dataclass serializer (I am overriding it to set custom field for SocialSecurityNumber dataclass):

class CustomDataclassSerializer(DataclassSerializer):
    serializer_field_mapping = {
        int:                  rest_framework.fields.IntegerField,
        float:                rest_framework.fields.FloatField,
        bool:                 rest_framework.fields.BooleanField,
        str:                  rest_framework.fields.CharField,
        decimal.Decimal:      fields.DefaultDecimalField,
        datetime.date:        rest_framework.fields.DateField,
        datetime.datetime:    rest_framework.fields.DateTimeField,
        datetime.time:        rest_framework.fields.TimeField,
        datetime.timedelta:   rest_framework.fields.DurationField,
        uuid.UUID:            rest_framework.fields.UUIDField,
        dict:                 rest_framework.fields.DictField,
        list:                 rest_framework.fields.ListField,
        SocialSecurityNumber: SocialSecurityNumberField
    }

    @property
    def serializer_dataclass_field(self):
        return CustomDataclassSerializer

The issue When this been invoked it tries for some reason to pass dataclass and many parameters to SocialSecurityNumberField (like it was a rest_framework.serializers.Serializer subclass), what I assume was not the expected behavior.

I get a validation error when passing "values": [""] into values: List[Optional[str]], "ErrorDetail(string='This field may not be blank.'", "code='blank')

Also is there a way to express list can't be empty?

The update method will not work with the PATCH method and partially updating.
Error: TypeError: __init__() missing 3 required positional arguments: 'id', 'port', and 'cipher'
It lists all the missing attributes and says they're required to init the dataclass.

When using many=True in a Serializer Generated by DataclassSerializer, the Optional Field (the id field in the example below) will have value of <class 'rest_framework.fields.empty'>.

In the example below the expected value is None since it is the default value:

@dataclass
class HelloWorldManyExample:
    name: str
    mobile_number: str
    id_only_optional: Optional[int]
    id_only_default: int = None
    id_default_and_optional: Optional[int] = None


class HelloWorldManyInput(DataclassSerializer):
    class Meta:
        dataclass = HelloWorldManyExample


class HelloWorldManyViewDataclass(APIView):
    parser_classes = (CamelCaseJSONParser,)
    renderer_classes = (CamelCaseJSONRenderer,)

    @swagger_auto_schema(request_body=HelloWorldManyInput(many=True))
    def post(self, request):
        hello_world_many_data = HelloWorldManyInput(data=request.data, many=True)
        hello_world_many_data.is_valid(raise_exception=True)
        return JsonResponse({"received": [asdict(data) for data in hello_world_many_data.validated_data]})

I have a project running the example above with swagger, you can check here in case that helps - https://github.com/dineshtrivedi/django-architecture/tree/drf-dataclass-issue-1

What do you think?

Hi there!

We are starting using this package a lot in our product and it's great!
One gap we reached into is support for dataclass union (typing.Union[dataclass1, dataclass2, ...]) as one of the dataclass fields.

Example case:

@dataclass
class A:
    a: int


@dataclass
class B:
    b: int


@dataclass
class Response:
    obj: A | B

class ResponseSerializer(DataclassSerializer)
    class Meta:
        dataclass = Response

I solved this by extending DataclassSerializer and add support using rest-polymorphic.
We also use drf-spectacular for openapi scheme and it also supports rest-polymorphic, so all good!

Before I open PR I wanted to check that this feature is useful, I would be happy to get a feedback :)

Environment

python==3.8
Django==3.2.4
djangorestframework==3.12.4
djangorestframework-dataclasses==0.9
dataclasses-json==0.5.4
drf-yasg==1.20.0

Problem

DataclassSerializer shows Dataclass name appears as "Dataclass" string instead of class name.

According to issue #14, it seems that even nested fields have proper names in the openapi format.

#serializer for response
@dataclass
@dataclass_json
class JobInfo:
    name: str
    labels: Dict[str, str]
    status: str
    accelerator: str
    limits: float
    requests: float
    volumes: List[str]
    image: str
    command: str
    startTime: str
    completionTime: str


@dataclass
@dataclass_json
class JobInfos:
    jobNumber: int
    jobs: List[JobInfo]


class JobInfoGetResponseSerializer(DataclassSerializer):
    class Meta:
        dataclass = JobInfos

Bonus points if they can also function as good usage examples.

Hi,

How about performance ?
Did you test large objects serialization performance ?

djangorestframework-dataclasses vs build-in drf serializer ?

Hello there!

We have a type alias that makes use of unions, and we would like to map it to a JSONField. This example summarizes the use case (simplified):

import typing as t
from dataclass import dataclass

JsonPrimitive = str | int | float | bool | None
Json = dict[str, "Json"] | list["Json"] | JsonPrimitive

class ExtendedDataclassSerializer(DataclassSerializer):
    serializer_field_mapping = DataclassSerializer.serializer_field_mapping | {
        Json: serializers.JSONField, 
    }


@dataclass
class Event:
    id: UUID

    @classmethod
    def get_serializer(cls):
        """Dynamically creates the serializer"""
        class Meta:
            dataclass = cls
        return type(
            f"{cls.__name__}Serializer", (ExtendedDataclassSerializer,), {"Meta": Meta}
        )

# ... many events inherit ...

@dataclass
class Event123(Event):
    config: Json

The current implementation does not support dynamically mapping fields of type Json to a JSONField. Note that this problem does not exist when declaring the Serializer as you can declare the config field as a JSONField. This problem affects only dynamic fields.

The following two patches made it work. I think the first one is a bug as it detects wrongly int | str | None as an Optional[int].

def is_optional_type_patch(tp: type) -> bool:
    """
    Patch original typing_utils.is_optional_type.

    This patch returns fixes detecting "None | int | str" as optional,
    only stuff like "None | int" should be detected as so.
    """
    origin = typing_utils.get_origin(tp)
    args = list(set(typing_utils.get_args(tp)))
    none_type = type(None)
    return (
        # int | None
        origin in typing_utils.UNION_TYPES
        and len(args) == 2
        and none_type in args
    ) or (
        # Literal["a", "b", None]
        typing_utils.is_literal_type(tp)
        and None in typing_utils.get_literal_choices(tp)
    )

typing_utils.is_optional_type = is_optional_type_patch


_original_looup_type_in_mapping = field_utils.lookup_type_in_mapping
def lookup_type_in_mapping_patch(mapping: dict[type, T], key: type) -> T:
    """
    This patch allows using anything as type annotation
    """
    if key in mapping:
        return mapping[key]
    return _original_looup_type_in_mapping(mapping, key)

field_utils.lookup_type_in_mapping = lookup_type_in_mapping_patch

The second patch allows mapping UnionType aliases to Field serializers but does not support overriding other kinds of aliases. For instance, it does not allow mapping JsonKV to a JSONField:

JsonPrimitive = str | int | float | bool | None
JsonKV = dict[str, JsonPrimitive]

@dataclass
class EventZ(Event):
    config: JsonKV

In this case, the library calls lookup_type_in_mapping using JsonPrimitive rather than JsonKV. To be honest, I don't know which is the best approach to solve this or even if this should be supported. It makes sense for our dynamic serializers though.

First of all, thanks for your work on this package. I've been using it successfully, but I've noticed and always ignored the fact that the .is_valid() call on the serializer returned False. I'd like to fix this to be able to take validation seriously, so I tried to find the issue. I may be doing something wrong, in this case, thanks for pointing me towards my mistake and forgiving me for having created this issue.

A minimal example based on your README (e.g., one can copy-paste this into the manage.py shell):

import datetime
import typing

from dataclasses import dataclass
from rest_framework_dataclasses.serializers import DataclassSerializer

@dataclass
class Person:
  name: str
  email: str
  alive: bool
  gender: typing.Literal['male', 'female']
  birth_date: typing.Optional[datetime.date]
  phone: typing.List[str]
  movie_ratings: typing.Dict[str, int]

serializer = DataclassSerializer(
  dataclass=Person,
  data=Person(name="Foo",
    email="[email protected]",
    alive=True,
    gender="male",
    phone=["000"],
    birth_date=None,
    movie_ratings={}))

Then, in the same django shell:

>>> serializer.is_valid()
False
>>> serializer._errors
{'non_field_errors': [ErrorDetail(string='Invalid data. Expected a dictionary, but got Person.', code='invalid')]}

This error is created when the DataclassSerializer is calling DRF's serializer's to_internal_value(data) method:

The validated data is still correct and can be sent out in a Response after having called is_valid(), but that's of course not how it's supposed to be used.

Thanks for having a quick look into this!

When using a serializer class and passing it a list of dataclasses I get the following error:

"Invalid data. Expected a dictionary, but got <dataclass.type>"

Example:

@DataClass
Module:
name: str

modules = [Module(name='foo'), Module(name='bar')]

class ModuleSerializer(DataclassSerializer):
class Meta:
dataclass = Module

ModuleSerializer(data=modules, many=True)

This requires me to first have to call:

modules_as_dicts = list(map(dataclasses.asdict, modules))

before passing the data to the serializer.

Seems odd for the serializer to not know how to access the dataclass data directly or use asdict internally?

I'm not sure if there's actually a solution here - but I came across this situation today where my usage of typing-only imports causes the dataclass type evaluator to fail.

I'd be open to submitting a PR for a patch (if possible) or a PR to provide better error messaging if this is a wont-fix.

Minimal example:

# app/foo/domains.py

from dataclasses import dataclass

@dataclass
class Foo:
    a: str

# app/bar/domains.py

from __future__ import annotations
from typing import TYPE_CHECKING
from dataclasses import dataclass

if TYPE_CHECKING:
    from app.foo.domains import Foo

@dataclass
class Bar:
    b: str
    foo: Foo

# app/bar/serializers.py

from rest_framework_dataclasses.serializers import DataclassSerializer
from app.bar.domains import Bar

class FooSerializer(DataclassSerializer):
    class Meta:
        dataclass = Foo

class BarSerializer(DataclassSerializer):
    foo = FooSerializer()

    class Meta:
        dataclass = Bar

Error (when calling to_representation on BarSerializer:

  ...
  File ".pyenv/versions/3.8.13/lib/python3.8/typing.py", line 1232, in get_type_hints
    value = _eval_type(value, base_globals, localns)
  File ".pyenv/versions/3.8.13/lib/python3.8/typing.py", line 270, in _eval_type
    return t._evaluate(globalns, localns)
  File ".pyenv/versions/3.8.13/lib/python3.8/typing.py", line 518, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'Foo' is not defined

👋 I have a dataclass that look like this:

@dc
class Document:
    id: UUID
    details: Union[ComplianceDoc, SignatureDoc]
    ...

@dc
class ComplianceDoc:
    ...

@dc
class SignatureDoc:
    ...

What is the expected behavior if I give this to a serializer? Currently it is just getting the first type within the Union.

The OpenAPI generator used here is drf-spectacular.

If a dataclass has an enum member, there should be a ChoicesField automatically generated for it.

https://docs.python.org/3/library/enum.html
https://www.django-rest-framework.org/api-guide/fields/#choicefield

I don't like the exceptions you otherwise get. We can set max_digits to None, but a sane default for decimal_places is harder. Maybe see if Django or DRF has a related setting we can use as default, or introduce one ourselves or something like that.

Here is a minimal example:

from dataclasses import dataclass

from rest_framework.fields import CharField
from rest_framework_dataclasses.serializers import DataclassSerializer


@dataclass
class Foo:
    bar: str


class ASerializer(DataclassSerializer):
    class Meta:
        dataclass = Foo
        fields = ("renamed_bar",)

    renamed_bar = CharField(source="bar")


ser = ASerializer(data={"renamed_bar": "string"})

ser.is_valid(raise_exception=True)

foo = ser.create(ser.validated_data)

This code does not raise on is_valid(raise_exception=True) but fails with KeyError when calling create.

Can someone provide any performance benchmark of ModelSerializer vs DataclassSerializer ?
take a look: https://hakibenita.com/django-rest-framework-slow

Fyi. @hakib

Hi I red the coment below in instantiate_data_dictionaries:

# I'm not sure this is actually the best way to deserialize into nested dataclasses. The cleanest way seems to
        # be overriding to_internal_value(), but the top-level serializer must return a dictionary from that method. We
        # could split nested dataclasses off into a separate DataclassField (which could in general clean-up the code a
        # bit), but that breaks specifying nested serializers using a single class. Let's use this ugly hack until I can
        # think of something better.

First i want to congratulate you, with dataclasses django and python are becoming much better a robust to deploy productions services, I'm also front end developer and is amazing how with typescript and angular you can prevent a lot of bugs with de interfaces, I have been looking away to replacate that in backend, i tryed TypedDict, but my code steel been very dificult to debug, so when I started to use dataclasses and your library, the enpoint works like magic.

So I was wondering or I don't understand why you dont instance the main Dataclass instead of creating a Dict, if you want I can try to create this improvement.

Thanks again for your amazing work!

Problem

The generated nested dataclass serializers are called DataclassSerializer, making it hard to differentiate them. When used with drf-spectacular this leads to unfortunate collisions that override the different types.

OrderedDict([('messages',
              MessageSerializer(many=True):
                  contacts = ContactSerializer(many=True):
                      name = CharField()
                  id = CharField()
                  message_timestamp = DateTimeField()
                  attachments = DataclassSerializer(many=True):
                      id = UUIDField()
                      filename = CharField()
                      url = CharField()),
             ('firm',
              DataclassSerializer(dataclass=<class 'fund_admin.comms.domain.FirmDomain'>):
                  id = IntegerField()
                  firm_name = CharField()
             ('entity',
              DataclassSerializer(dataclass=<class 'fund_admin.comms.domain.EntityDomain'>):
                  id = IntegerField()
                  entity_name = CharField()

This is what get_fields returns for my serializer. As you can see, the generated DataclassSerializers have the name DataclassSerializer. When this is put through drf-spectacular, this causes the Components to all be called "Dataclass" (Serializer postfix is stripped) and the last one to be evaluated (most nested one in this case) overrides the type in the schema.

Proposal: Prefix serializer class name with dataclass name

On line 512 in rest_framework_dataclasses/serializers.py

def build_dataclass_field(self, field_name: str, type_info: TypeInfo) -> SerializerFieldDefinition:
...
    # add this line at 512
    class NewClass(field_class):
        pass
    NewClass.__name__ = type_info.base_type.__name__ +  "Serializer"
    return NewClass, field_kwargs

Generated schema showing that attachments override FirmDomain and EntityDomain

    Dataclass:
      type: object
      properties:
        id:
          type: string
          format: uuid
        filename:
          type: string
        url:
          type: string
      required:
      - filename
      - id
      - url

cc: @tfranzel
@oxan

Hello, I am working on deserializing Jira user lists. The gist of the code looks like this:

from dataclasses import dataclass

from rest_framework_dataclasses.serializers import DataclassSerializer

from integrations.jira.serializers.common import UserBasic

@dataclass
class User:
    displayName: str

class UserSerializer(DataclassSerializer):
    class Meta:
        dataclass = User

serializer = UserSerializer(data=[{"displayName": "a"}], many=True)
serializer.is_valid()
users = serializer.save()

The last save raises.

Traceback (most recent call last):
  File "/opt/src/integrations/tests/test_jira_serializers_inbound.py", line 19, in test_UserSerializer_deserialize
    users = serializer.save()
            ^^^^^^^^^^^^^^^^^
  File "/opt/virtualenv/lib/python3.11/site-packages/rest_framework/serializers.py", line 698, in save
    validated_data = [
                     ^
  File "/opt/virtualenv/lib/python3.11/site-packages/rest_framework/serializers.py", line 699, in <listcomp>
    {**attrs, **kwargs} for attrs in self.validated_data
    ^^^^^^^^^^^^^^^^^^^
TypeError: 'User' object is not a mapping

attrs in the Django Rest Framework is a dataclass so it is not possible to unwrap it.

The dependency versions:
django==3.2.12
djangorestframework==3.12.4
djangorestframework-dataclasses==1.2.0

How can I help?

Hi!

I found some bug and some fix :)

The current version doesn't work with serializers like that:

class MyDataclassSerialzer1(...):
   ...
class MyDataclassSerializer2(...):
   my_field = MyDataclassSerialzer1(many=True)
   ...

The fix is simple! Just change this line:

to:

            if isinstance(field, DataclassSerializer) or \
                    isinstance(field, rest_framework.serializers.ListSerializer) and \
                    isinstance(field.child, DataclassSerializer):

Someday I will have a time to make PR with tests, but not today :(

When using code like:

@dataclass
class MyDataclass:
    blah: int


class MySerializer(DataclassSerializer[MyDataclass]):
    ...

mypy version 0.990 will display a warning like:

Type argument "MyDataclass" of "DataclassSerializer" must be a subtype of "Dataclass"

I am not yet sure whether this is a bug or intentional change, reported to mypy for clarification: python/mypy#14029

Environment

Python 3.8

Django==3.2.5
djangorestframework==3.12.4
djangorestframework-dataclasses==0.9
drf-yasg=1.20.0

Problem

After upgrading from 0.8 to 0.9 I've got the following exception:

Internal Server Error: /swagger/
Traceback (most recent call last):
  File "/.../venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/.../venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/.../venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/.../venv/lib/python3.8/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/.../venv/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/.../venv/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/.../venv/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/.../venv/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/views.py", line 94, in get
    schema = generator.get_schema(request, self.public)
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/generators.py", line 246, in get_schema
    paths, prefix = self.get_paths(endpoints, components, request, public)
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/generators.py", line 404, in get_paths
    operation = self.get_operation(view, path, prefix, method, components, request)
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/generators.py", line 446, in get_operation
    operation = view_inspector.get_operation(operation_keys)
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/view.py", line 45, in get_operation
    responses = self.get_responses()
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/view.py", line 182, in get_responses
    responses=self.get_response_schemas(response_serializers)
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/view.py", line 269, in get_response_schemas
    schema=self.serializer_to_schema(serializer),
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/base.py", line 437, in serializer_to_schema
    return self.probe_inspectors(
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/base.py", line 110, in probe_inspectors
    result = method(obj, **kwargs)
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/field.py", line 33, in get_schema
    return self.probe_field_inspectors(serializer, openapi.Schema, self.use_definitions)
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/base.py", line 228, in probe_field_inspectors
    return self.probe_inspectors(
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/base.py", line 110, in probe_inspectors
    result = method(obj, **kwargs)
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/field.py", line 124, in field_to_swagger_object
    actual_schema = definitions.setdefault(ref_name, make_schema_definition)
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/openapi.py", line 688, in setdefault
    ret = maker()
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/field.py", line 100, in make_schema_definition
    child_schema = self.probe_field_inspectors(
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/base.py", line 228, in probe_field_inspectors
    return self.probe_inspectors(
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/base.py", line 110, in probe_inspectors
    result = method(obj, **kwargs)
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/inspectors/field.py", line 640, in field_to_swagger_object
    choice = field_value_to_representation(field, choice)
  File "/.../venv/lib/python3.8/site-packages/drf_yasg/utils.py", line 461, in field_value_to_representation
    value = field.to_representation(value)
  File "/.../venv/lib/python3.8/site-packages/rest_framework_dataclasses/fields.py", line 37, in to_representation
    return value.value
AttributeError: 'str' object has no attribute 'value'

I use a subclass of models.TextChoices as a type in my model and as a type for the dataclass.

Example

constants.py

class PetType(models.TextChoices):
    CAT = 'cat', _('Cat')
    DOG = 'dog', _('Dog')
    PARROT = 'parrot', _('Parrot')
    HAMSTER = 'hamster', _('Hamster')

models.py

class Pet(models.Model):
    name = models.CharField(max_length=30)
    type = models.CharField(max_length=7, choices=PetType.choices)
    age = models.PositiveSmallIntegerField()

serializers.py

@dataclass
class PetData:
    name: str
    age: int
    type: PetType


class PetSerializer(DataclassSerializer):
    class Meta:
        dataclass = PetData

I am experiencing some behaviors that seem to be weird for me. The case is mainly when I use Optional and defaults values in my dataclass and pass it in the class Meta. I started to suspect when I used the Serializer using DataclassSerializer in swagger.

Here is a code example with comments:

@dataclass
class HelloWorld:
    first_name: str
    surname: str
    age: int

    # Optional[float] is not required on payload and swagger, but fails on hello_world_data.validated_data because it
    # is not provided and has no default value
    # FIXME: This should be required still in the serializer object but with allow_null=True
    height_only_optional: Optional[float]

    # float = None - Required in the payload and swagger, so the default value does not make any difference
    # FIXME: Should we make required as False when the dataclass has default value?
    height_none_default: float = None

    # Optional[float] = None - Not required in the payload and swagger, and height was set to None. This worked :)
    # FIXME: Should this add default=None to serializer?
    height_option_and_none_default: Optional[float] = None

    # FIXME: Should the rule be?
    # Optional is always allow_null=True (and required=True), but it is required=False only if there is a default value
    # Require=False is not set in any other case, but with a default value
    # Add has_default_value to TypeInfo and make the field required=False if it has a default value
    # Why?
    # required=False seems to be complicated on the dataclass since in the serializer it means the data won't necessarily be there
    # but for the dataclass to be instantiated successfully the data needs to be there unless it has a default value


class HelloWorldInput(DataclassSerializer):
    """
    >>> print(repr(HelloWorldInput()))
    HelloWorldInput():
        first_name = CharField()
        surname = CharField()
        age = IntegerField()
        height_only_optional = FloatField(allow_null=True, required=False)
        height_none_default = FloatField()
        height_option_and_none_default = FloatField(allow_null=True, required=False)
    """
    class Meta:
        dataclass = HelloWorld

You can read the FIXME comments, but the summary of the rules that seems to be right for me are these:

 # FIXME: Should the rule be?
# Optional is always allow_null=True (and required=True), but it is required=False only if there is a default value
# Require=False is not set in any other case, but with a default value
# Add has_default_value to TypeInfo and make the field required=False if it has a default value

I have a project running the example above with swagger, you can check here in case that helps - https://github.com/dineshtrivedi/django-architecture/tree/drf-dataclass-issue-1

What do you think? Does it make sense what I am proposing?

It would be nice to write:

@model
@dataclass
class Person:
    name: str
    email: str
    alive: bool
    gender: typing.Literal['male', 'female']
    birth_date: typing.Optional[datetime.date]
    phone: typing.List[str]
    movie_ratings: typing.Dict[str, int]

and have this part auto generated:

class PersonSerializer(DataclassSerializer):
    class Meta:
        dataclass = Person

would you consider such a pull request?

Apparantly Travis don't have Python 3.9 yet... However, I'm in general not too happy with Travis, so might be a good moment to figure out GitHub Actions.

Hi,
given the following:

@dataclass
class Person():
  name: str


class PersonSerializer(DataclassSerializer):
  class Meta:
     dataclass = Person

When I have my json data from the client, I can serialize it with serialized_data = PersonSerializer(data={'name':'alice'}). However, if I call is_valid() and then validated_data, this will give me an OrderedDict, instead of a Person object. Am I missing something obvious, or do I just have to call Person(**serialized_data) to get it back to my Person dataclass?

Hi,
what is the motivation behind converting an optional field into allow_null=True instead of using required=False, allow_null=True, allow_blank=True? Even having an empty string does raise an error when validating an optional field.

As @intgr mentioned in #30, it might be nice to support specifying the serializer field configuration using dataclass field metadata.

I've thought about this a bit. It would be helpful and much cleaner if one could pass custom field kwargs overrides using dataclass field(metadata=...), e.g.:

@dataclass
class Example:
    string: str = field(metadata={"drf_field_args": {"allow_blank": True}})
    lst: List[str] = field(metadata={"drf_field_args": {"child": serializers.CharField(allow_blank=True)}})

Does this seem like a mechanism you want to support @oxan?

Or perhaps a metadata key that allows overriding the field as a whole

@dataclass
class Example:
    string: str = field(metadata={"drf_field": serializers.CharField(allow_blank=True)})
    lst: List[str] = field(metadata={
        "drf_field": serializers.ListField(child=serializers.CharField(allow_blank=True))
    })

adapted sample

@dataclass
class MaybeBalls:
    balls: Optional[List[str]]

class MaybeBallsSerializer(DataclassSerializer):
    class Meta:
            dataclass = MaybeBalls

input = {}
serializer = MaybeBallsSerializer(data=input)
print(serializer.data) # will fail

yields something like

  File "/usr/local/lib/python3.8/site-packages/rest_framework/serializers.py", line 632, in data
    ret = super().data
  File "/usr/local/lib/python3.8/site-packages/rest_framework/serializers.py", line 320, in data
    self._data = self.to_representation(self.validated_data)
  File "/usr/local/lib/python3.8/site-packages/rest_framework/serializers.py", line 599, in to_representation
    ret[field.field_name] = field.to_representation(attribute)
  File "/usr/local/lib/python3.8/site-packages/rest_framework/fields.py", line 1692, in to_representation
    return [self.child.to_representation(item) if item is not None else None for item in data]
TypeError: 'type' object is not iterable

it tries to iterate over a <serializer.empty>