Feature request: MapField about django-mongodbforms

[jschrewe]

Probably easy. Sadly the mongoengine documentation is a bit short on them. Do you know if the key value must be a string? Or can it be any type? The actual value associated with them is clearly defined, as in passed as the field argument to the field, right?

[laurentpayot]

I did a quick test by adding a MapField with interger keys to one of my models and when I tried to save it I got: "Invalid dictionary key - documents must have only string keys". And yes the type of value associated with them is defined once for all in the model definition, e.g. illustrations = MapField(ImageField(size=(600, 800, True)))

[jschrewe]

Done. Not really tested though. Actually not tested at all except for my dev example using an IntField. Kay and value fields are wrapped in a <fieldset> which seems to be the only tag that is there to group forms elements belonging together.

[laurentpayot]

Amazing. I will test it asap.

[laurentpayot]

After modifications described in #38 it looks like it's working. I'm trying to make it work with a MapField of ImageField and I get weird 'InMemoryUploadedFile' object has no attribute 'grid_id' errors. Nothing to do with mongodbforms I think. So I need to do more testing...

[laurentpayot]

I did some more tests and it's working great with a MapField of StringField for instance.
But with a MapField of ImageField here's what I get (I'm using the latests commits of pymongo, mongoengine, and django):

Traceback (most recent call last):
  File "/usr/local/lib/python3.3/dist-packages/django/core/handlers/base.py", line 114, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.3/dist-packages/django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.3/dist-packages/django/utils/decorators.py", line 29, in _wrapper
    return bound_func(*args, **kwargs)
  File "/usr/local/lib/python3.3/dist-packages/django/contrib/auth/decorators.py", line 25, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.3/dist-packages/django/utils/decorators.py", line 25, in bound_func
    return func(self, *args2, **kwargs2)
  File "/opt/publiberty/django/www/views.py", line 425, in dispatch
    return super().dispatch(*args, **kwargs)
  File "/usr/local/lib/python3.3/dist-packages/django/views/generic/base.py", line 87, in dispatch
    return handler(request, *args, **kwargs)
  File "/usr/local/lib/python3.3/dist-packages/django/views/generic/edit.py", line 228, in post
    return super(BaseUpdateView, self).post(request, *args, **kwargs)
  File "/usr/local/lib/python3.3/dist-packages/django/views/generic/edit.py", line 170, in post
    if form.is_valid():
  File "/usr/local/lib/python3.3/dist-packages/django/forms/forms.py", line 128, in is_valid
    return self.is_bound and not bool(self.errors)
  File "/usr/local/lib/python3.3/dist-packages/django/forms/forms.py", line 120, in errors
    self.full_clean()
  File "/usr/local/lib/python3.3/dist-packages/django/forms/forms.py", line 274, in full_clean
    self._post_clean()
  File "/usr/local/lib/python3.3/dist-packages/mongodbforms-0.1.5-py3.3.egg/mongodbforms/documents.py", line 362, in _post_clean
    f.validate(value)
  File "/usr/local/lib/python3.3/dist-packages/mongoengine-0.8.2-py3.3.egg/mongoengine/fields.py", line 761, in validate
    super(DictField, self).validate(value)
  File "/usr/local/lib/python3.3/dist-packages/mongoengine-0.8.2-py3.3.egg/mongoengine/base/fields.py", line 354, in validate
    self.field._validate(v)
  File "/usr/local/lib/python3.3/dist-packages/mongoengine-0.8.2-py3.3.egg/mongoengine/base/fields.py", line 176, in _validate
    self.validate(value, **kwargs)
  File "/usr/local/lib/python3.3/dist-packages/mongoengine-0.8.2-py3.3.egg/mongoengine/fields.py", line 1240, in validate
    if value.grid_id is not None:
AttributeError: 'InMemoryUploadedFile' object has no attribute 'grid_id'

ImageFields are working nicely with mongodbforms when they're outside of a MapField.
Any idea about where it could come from?

[jschrewe]

I think it has to do with mongodbforms actually. My guess would be that the forms parts has to look inside the mapfield (and listfield probably too) and handle the save correctly. That should already be done for normal file fields but not for "wrapped" filefields.

[laurentpayot]

I mainly use MapFields with ImageFields, I will owe you a beer if you can make it work ;)

[jschrewe]

So how do you usually add files to your map fields? Because the code I have now should work in theory but doesn't in practice because apparently I'm too stupid to add an image and save the whole thing.

[jschrewe]

So now after da4b134 it saves. In a really, really ugly way though. I also opened an issue at Mongoengine to see if it really has to be that ugly here.

[laurentpayot]

The only way I found to add images to my "illustrations" MapField is to create a dummy BookIllustration model that's only used to get an image object in a "set_illustration" function of my Book model. That is the ugliest code in the world:

# models.py
class BookIllustration(Document):
    image = ImageField(size=(600, 800, True))
class Book(Document):
    def set_illustration(self, title, file):
        title = title.strip() # title cleaning
        if not title:
            raise ValidationError("You can't add an illustration without a title.")
        img = BookIllustration(image=file).image # workaround needed to use MapField(ImageField())!!!
        self.illustrations[title] = img

Then the images are added/deleted in the form_valid() function of my view that inherits from FormView with form_class = BookIllustrationFormSet

BookIllustrationFormSet is a custom formset:

# forms.py
class BaseBookIllustrationFormSet(BaseFormSet):
    """Custom base formset for custom cleaning."""
    
    def clean(self):
        """Checks that no two illustrations have the same title."""
        if any(self.errors):
            # don't bother validating the formset unless each form is valid on its own
            return
        titles = []
        for i in range(0, self.total_form_count()):
            form = self.forms[i]
            title = form.cleaned_data.get('title', '').strip()
            if title in titles:
                raise ValidationError(_("You can't use the same title for different illustrations."))
            titles.append(title)
class BookIllustrationForm(TitleRequiredMixin, forms.Form):
    """Book Illustration form used in BoookIllustrationFormSet."""
    title = forms.CharField(max_length=100, label=_("title"),
                            required=False, widget=forms.TextInput(attrs={'class':'book_input'}))
    file = forms.ImageField(label = _("file"), required=False)
BookIllustrationFormSet = formset_factory(BookIllustrationForm, formset=BaseBookIllustrationFormSet,
                                          can_delete=True)

(Note the can_delete=True, that's a sweet feature that could be used in mongodbforms too.)

I'm going to check your last commit, it will always be better than my solution.

[jschrewe]

Good to see you ran into the same troubles. Or not. Well you know what I mean.

There still is a problem with adding images – the first save adds a new element to the dict, the second save actually adds an image – otherwise it seems to work just fine now.

The can_delete looks nice. Not sure why it's ignored though because I know we pass it through to Django's classes.

[jschrewe]

Seems to work for me now. I still think Mongoengine's file handling is horrible and buggy but until they change it this should do it.

[laurentpayot]

I opened #39 and #40 to keep you in shape.

[laurentpayot]

Anyway even without MapField of ImageFields mongodbforms is a great piece of code. I'm going to use it a lot. Thanks a lot for it.