grantmcconnaughey / django-field-history Goto Github PK

Hi.

There is a problem with Models which have primary key with default=uuid

id = models.UUIDField(default=uuid.uuid4, editable=False, primary_key=True)

The initial/first field history value is not saved to database.
It is caused by is_new_object = instance.pk is None in tracker.py

This approach has a problem:
In the save method, self.pk will never be None when default=uuid.uuid is set.

A large number of django users (and a lot of non-official django tutorials) believe that checking self.pk in the save method is a safe way to detect and decide whether an instance of a model is new or not.

No its not. The safe way to detect and decide whether an instance of a model is new or not is to use self._state object
https://docs.djangoproject.com/en/3.2/ref/models/instances/#state

So I think you should switch from
is_new_object = instance.pk is None
to
is_new_object = instance._state.adding

Just an issue to keep track of the discussion in #25 (comment)

Hi,

It's possible track all the fields of the model?

Thanks

Is there any way we can temporarily disable tracking either by disconnecting signals or using some magic settings?
Example reason for doing this would be to pull historical data (timestamp, value) from already existing source. I'm not sure this is possible at this point..

@grantmcconnaughey I'll be glad to make PR + docs, after we figure out the way to handle this.
Thanks!

$ ./manage.py createinitialfieldhistory
Traceback (most recent call last):
File "src/manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 399, in execute_from_command_line
utility.execute()
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 272, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 75, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/utils/importlib.py", line 40, in import_module
import(name)
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/field_history/management/commands/createinitialfieldhistory.py", line 4, in
from django.apps import apps
ImportError: No module named apps

I use a uuid for a primary key on my objects, and as such call it uuid. When I go to create the initial field history, I get this error:

File "lib/python2.7/site-packages/field_history/management/commands/createinitialfieldhistory.py", line 35, in handle object_id=obj.id,
AttributeError: 'Order' object has no attribute 'id'

I fixed this by changing line 35 in createinitialfieldhistory.py to object_id=obj.pk

In postgresql there is specific and optimized field for JSON, jsonb which can permit the direct used of the json object (and i guess that other dbb has his own).
The created table stock in serialized_ data the object. I think it could really improve the lib efficiency if the field type could set himself regarding project configuration.

Consider the following models:

class OKB_Article(models.Model):
    VISUALIZATION_CHOICES = (
        ('is_private', 'Privado'),
        ('is_public', 'Publico'),
    )
    id_record = models.BigIntegerField()
    title = models.CharField(max_length=200)
    visualization = models.CharField(
        max_length=50, choices=VISUALIZATION_CHOICES, default='is_private')
    article_grouping = models.ForeignKey(
            OKB_ArticleGrouping,
            blank=True,
            null=True)
    author = models.ForeignKey(User, related_name='user_author')
    motive_type = models.ForeignKey(OSD_Motive, blank=True, null=True)
    article_type = models.ForeignKey(OKB_ArticleType, blank=True, null=True)
    content = models.TextField()
    approved_by = models.ForeignKey(
            User, related_name='approved_by', blank=True, null=True)
    created_date = models.DateTimeField(auto_now_add=True)
    status_type = models.ForeignKey(OSD_Status)

    history = FieldHistoryTracker([
        'motive_type', 'article_grouping', 'article_type',
        'visualization', 'status_type'
        ])

    class Meta:
        db_table = 'okb_article'

    def __unicode__(self):
        return self.title

class OSD_Status(models.Model):
    name = models.CharField(max_length=50)
    color = models.CharField(max_length=6, choices=COLOR_CHOICES,
                             default="ffce93")
    description = models.CharField(max_length=200)
    behavior = models.ForeignKey(OSD_Behavior)
    motive = models.ManyToManyField(OSD_Motive, through='OSD_StatusMotive')
    status = models.BooleanField(default=True)

    class Meta:
        db_table = 'osd_status'

    def __unicode__(self):
        return "%s - %s" % (self.name, self.behavior)

As you can see the field status_type in OKB_Article has a ManyToManyField called motive.

If I'm tracking the status_type field, the field_value function only works for that field; I can't access the field_value of any other field, it throws a DeserializationError mentioning the status_type field even though I'm not accesing it. See the screenshot where I reproduce and mimick what the field_value function does inside an ipdb debugger:

However, this only happens when I'm tracking said field. If I remove it from the FieldHistoryTracker list, I can access all the other fields without a problem.

I'm using the latest version of Django 1.9.x along with the latest Python 2.7.x on Linux.

Django 1.10 using MIDDLEWARE instead of MIDDLEWARE_CLASSES in settings.py. See https://docs.djangoproject.com/en/1.10/releases/1.10/#new-style-middleware for details.

missing this feature on the project, which could be very usefull. Or at last the documentation must explain how the history table looks like and his behavior. Anyway good job and good luck/

I faced with the problem, when I want to track fields of parent model using django-field-history app i'm unable to save the value of the field due to the problem of how Django serializers works. They all returns copy just of tables, but working with inherited models we need to search among parent fields too.
As result data field doesn't contains any fields values.

We had an issue in https://github.com/twschiller/open-synthesis/issues/214 where we'd get a database integrity error about the FieldHistory referencing users that didn't exist

We had to add the following to our base classes to reset the request context after tests where we used the Django test client with a logged in user

 def tearDown(self) -> None:
        FieldHistoryTracker.thread.request = None

Not a huge problem but it would be good to have the installation and quickstart sections of the docs mention that you need to run manage.py migrate after adding 'field_history' to INSTALLED_APPS. Would save django newbies and people who don't spend a ton of time with it a minute or two of head-scratching, and is more technically complete.

Hello!

I am using PostgreSQL and I'm getting the following error upon python manage.py createinitialfieldhistory

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 28, in <module>
    execute_from_command_line(sys.argv)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/home/vagrant/grt/lib/python3.6/site-packages/field_history/management/commands/createinitialfieldhistory.py", line 37, in handle
    field_name=field).exists():
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/models/query.py", line 715, in exists
    return self.query.has_results(using=self.db)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/models/sql/query.py", line 516, in has_results
    return compiler.has_results()
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1031, in has_results
    return bool(self.execute_sql(SINGLE))
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1061, in execute_sql
    cursor.execute(sql, params)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "field_history_fieldhistory" does not exist
LINE 1: SELECT (1) AS "a" FROM "field_history_fieldhistory" WHERE ("...

my model:

class Title(models.Model):
    id = models.PositiveIntegerField(primary_key=True)
    name = models.CharField(max_length=255) 
    priority = models.PositiveIntegerField(null=True, blank=True)
    field_history = FieldHistoryTracker(['priority'])

tried doing this as well: https://django-field-history.readthedocs.io/en/latest/readme.html#working-with-mysql (just in case) - no success.
What could be wrong?

edit:
you have to use python manage.py migrate your new field first

I like your approach to recording model history. I've tried using django-reversion before but it broke as I migrated my models by adding or deleting fields. I wonder, have you considered how django-field-history would handle migrations?

I t's just my suggestion but I think, for PostgreSQL you shoud use fields like HStoreField or JSONField for serialized_data field.

• Django < 1.8.x: models.TextField()
• Django = 1.8.x: from django.contrib.postgres.fields import HStoreField https://docs.djangoproject.com/en/1.8/ref/contrib/postgres/fields/#hstorefield
• Django >= 1.9.x and PostgreSQL >= 9.4: from django.contrib.postgres.fields import JSONField https://docs.djangoproject.com/en/1.9/ref/contrib/postgres/fields/#jsonfield

I need to run python manage.py createinitialfieldhistory inside provision script during launch, so every time I deploy it creates new values for already tracked fields, which I don't think is necessary.

Is it possible for this command to omit already tracked objects? Thanks.

Please add function so you can take the user's which change instance from request

Please add keyword for track all fields, like a __all__

I would like to rename the field "status" to "portal_status" in my app. Originally this is what it looks like:

I executed python manage.py renamefieldhistory --model=accounts.account --from_field=status --to_field=portal_status and saw that 619 rows were updated.

Upon closer inspection however, I see that field_name is changed but not the field inside serialized_data.

This causes my application to crash.

Perhaps we should also parse serialized_data to and modify the keys accordingly?

InnoDB indexes in MySQL can only be created for columns with a max length of 767 bytes (see the third bullet under Maximums and Minimums), so the TextField that FieldHistory.object_id currently uses causes migration 0001 to throw an exception. Example output:

Operations to perform:
  Apply all migrations: field_history
Running migrations:
  Rendering model states... DONE
  Applying field_history.0001_initial...Traceback (most recent call last):
  File "project/proj/manage.py", line 8, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/site-packages/raven/contrib/django/management/__init__.py", line 41, in new_execute
    return original_func(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 222, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 110, in migrate
    self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 148, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 91, in __exit__
    self.execute(sql)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 111, in execute
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 98, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
    return self.cursor.execute(query, args)
  File "/usr/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/usr/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
django.db.utils.OperationalError: (1170, "BLOB/TEXT column 'object_id' used in key specification without a key length")

The obvious solution is to change FieldHistory.object_id to a fixed-length field, such as a CharField.

This does not work, if object representation have type unicode:

return '{} field history for {}'.format(self.field_name, self.object)

fix for python 2.7:

return u'{} field history for {}'.format(self.field_name, self.object)

https://github.com/grantmcconnaughey/django-field-history/blob/master/field_history/models.py#L31

Hi!

I have a question. Your tracker uses threading.locals. Have you ever tried to test this decisiton on a production with more or less big number of users who make changes to your models at the same time?
If you use uwsgi then Django will be just forked by some number of processes (2, 4, 5, whatever you set in your uwsgi settings). So, it means that if you have 1000 online users then, e.g., and 4 forks of Django, then 250 of your 'requests' which you store in thread will go to the same locals and will overwrite each other.
To my mind, it's not serious to tell people that they should use this middleware in their project with full confidence that it will log all changes correctrly.