Add on_set / on_get callbacks to fields about umongo

[touilleMan]

Do you think the new marshmallow's pre/post processors integration (see #60 (comment)) could be used as a workaround for this ?

[lafrech]

Not sure. The on_[g|s]et callbacks could be executed after an attribute is set.

class User(Document):
    birthday = DateTimeField(on_set=self._compute_age)  # either here
    age = IntField()

    def _compute_age(self):
        self.age = (datetime.utcnow() - self.birthday).days / 365  # Who cares about precision?

chuck = User()
chuck.age = None

chuck.birthday = datetime.datetime(year=1940, month=3, day=10)
chuck.age = 76

I don't think you can do this with post_load, can you? AFAIU, post_load is schema-level, not field-level, it is only called when loading the whole schema (maybe with partial), but not when setting an attribute.

[touilleMan]

You're right, marshmallow's pre/post processors is not the silver bullet we were waiting for !

I'm thinking of a new way to implement this feature:

• on_set should be simple by adding a check in Dataproxy.load, Dataproxy.set and Dataproxy.set_by_mongo_name
• on_get can be implemented by storing the field data in dataproxy as a lazy object that need evaluation

example:

>>> def capitalize(value):
... print('capitalize !')
... return value.to_upper()
... 
>>> def double(value):
... print('double !')
... return value + ' ' + value
... 
>>> class User(Document):
...     name = fields.StrField(on_get=[capitalize, double])
...     lastname = fields.StrField(on_set=[capitalize, double])
... 
>>> user = User(name='john', lastname='doe')
capitalize !
double !
>>> user.lastname
'DOE DOE'
>>> user.name
capitalize !
double !
'JOHN JOHN'
>>> lazy_obj = user._data._data['name']
>>> lazy_obj
<LazyObj(fn=double, value=LazyObj(fn=capitalize, value='john'))>
>>> lazy_obj.eval()
capitalize !
double !
'JOHN JOHN'

But I'm still unsure how useful it would be for on_get... do you see a usecase for this ?
I can think of a way to automatically unserialize data (for example returning a dict when having a string of json in database). However this feels error-prone (in my example you may want to modified dict but this will change nothing within the database...)

Usecase for on_set is pretty obvious (hashing&salting password as you proposed) though.

[lafrech]

But I'm still unsure how useful it would be for on_get... do you see a usecase for this ?

I also see more use for on_set. I guess on_get came naturally for symmetry reasons. In practice, it could be useful for a field that depends on a lot of other fields with a simple computation. So rather than modifying the field each time one of those other fields is modified, it could be modified on the fly when accessed. This could be better in terms of both performance and code simplicity. But it doesn't have to be done in the same move anyway, it is independent from on_set.

on_set should be simple by adding a check in Dataproxy.load, Dataproxy.set and Dataproxy.set_by_mongo_name

And Dataproxy.update, Dataproxy.delete. I'm not sure about Dataproxy.from_mongoand Dataproxy._add_missing_fields.

Questions:

• Should the callbacks be called when loading from Mongo or only when setting values from object world?
• Should there be a mechanism to avoid calling several times the same callback in a load / update operation, but rather do all updates first then call all needed callbacks only once? Tempting, but maybe not that easy, and potentially problematic. The callbacks are not necessarily idempotent like in my age computation example. If the callback counts or adds/appends stuff, the designer might want to have them executed systematically.
• From an implementation perspective, rather than modifying all the methods cited above, maybe there is an alternative where Dataproxy._data would be a special kind of dict with an overriden __setitem__. Just a tought. I didn't check feasibility.