Issue with djangocms_googlemap_0005 migration: no such column about djangocms-googlemap

Tested and confirmed, works like a charm.

Using current PyPi (1.1.0):

(.venv-staging) 21:37 robert@MacBook-Pro-2:~/dev/XXXXXXX$ ./manage.py migrate djangocms_googlemap
Operations to perform:
  Apply all migrations: djangocms_googlemap
Running migrations:
  Rendering model states... DONE
  Applying djangocms_googlemap.0004_adapted_fields... OK
  Applying djangocms_googlemap.0005_create_nested_plugins...Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
[...]    
  File "/Users/robert/dev/XXXXXXX/.venv-staging/lib/python2.7/site-packages/pymysql/err.py", line 115, in _check_mysql_exception
    raise InternalError(errno, errorvalue)
django.db.utils.InternalError: (1054, u"Unknown column 'icon_id' in 'field list'")

Using fixes/63 branch:

(.venv-staging) 21:37 robert@MacBook-Pro-2:~/dev/XXXXXXX$ ./manage.py migrate djangocms_googlemap
Operations to perform:
  Apply all migrations: djangocms_googlemap
Running migrations:
  Rendering model states... DONE
  Applying djangocms_googlemap.0004_adapted_fields... OK
  Applying djangocms_googlemap.0005_create_nested_plugins... OK
  Applying djangocms_googlemap.0006_remove_fields... OK
  Applying djangocms_googlemap.0007_reset_null_values... OK
  Applying djangocms_googlemap.0008_removed_null_fields... OK
  Applying djangocms_googlemap.0009_googlemapmarker_icon... OK

Many thanks!

[FinalAngel]

hello @Striv thank you for the report. Would you be able to provide additional information such as:

• What database system are you using
• What Django / django CMS / django CMS Google Map Version is in use
• What operation system is the project running on

@czpython might you have some initial ideas?

[czpython]

Hello @Striv,
I believe we've found the issue but just to confirm, the project in question was using an older version of this app?

[Striv]

In response to @FinalAngel: Sqlite3, Djangocms_googlemap 0.5.2 to begin with, now using 1.1.0. After submitting a bug report to you on here, I went ahead and cleared out the DB table that was causing the migration error, re-migrated, which appears to go through.

I'm not sure how I would save that data off, if the model was changed somehow between versions that is no longer viable.

I suspect that this specific issue is resolved, although I do need to go searching additional help, because freeing up this one migration only works sometimes.

When the project is first installed and the first migrate is run, everything passes, but then it complains about no users in auth and my login doesnt work. Only once I notice the "unapplied migrations" message on runserver do I call migrations again, and the above strategy of clearing djangocms_googlemap_googlemap table out works.

Any thoughts?

Confirmed, issues with column missing. Something went wrong with building migrations, there is no reason why in my configuration migrations shouldn't work properly:

Running migrations:
  Rendering model states... DONE
  Applying djangocms_googlemap.0005_create_nested_plugins...Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
    utility.execute()
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 345, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 200, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/migrations/executor.py", line 92, in migrate
    self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/migrations/executor.py", line 121, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/migrations/executor.py", line 198, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/migrations/migration.py", line 120, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/migrations/operations/special.py", line 183, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/djangocms_googlemap/migrations/0005_create_nested_plugins.py", line 35, in create_marker_and_route
    info_content=gmap.content,
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/utils/decorators.py", line 184, in inner
    return func(*args, **kwargs)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/cms/api.py", line 372, in add_plugin
    plugin.save()
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/cms/models/pluginmodel.py", line 341, in save
    super(CMSPlugin, self).save(*args, **kwargs)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/models/base.py", line 708, in save
    force_update=force_update, update_fields=update_fields)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/models/base.py", line 736, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/models/base.py", line 801, in _save_table
    forced_update)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/models/base.py", line 851, in _do_update
    return filtered._update(values) > 0
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/models/query.py", line 645, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1149, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 848, in execute_sql
    cursor.execute(sql, params)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/utils.py", line 95, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 112, in execute
    return self.cursor.execute(query, args)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/pymysql/cursors.py", line 146, in execute
    result = self._query(query)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/pymysql/cursors.py", line 296, in _query
    conn.query(q)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/pymysql/connections.py", line 819, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/pymysql/connections.py", line 1001, in _read_query_result
    result.read()
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/pymysql/connections.py", line 1285, in read
    first_packet = self.connection._read_packet()
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/pymysql/connections.py", line 965, in _read_packet
    packet.check_error()
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/pymysql/connections.py", line 394, in check_error
    err.raise_mysql_exception(self._data)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/pymysql/err.py", line 120, in raise_mysql_exception
    _check_mysql_exception(errinfo)
  File "/Users/robert/dev/XXXXXXXXX-YYYY/.venv/lib/python2.7/site-packages/pymysql/err.py", line 115, in _check_mysql_exception
    raise InternalError(errno, errorvalue)
django.db.utils.InternalError: (1054, u"Unknown column 'icon_id' in 'field list'")

Details:

• mysql Ver 14.14 Distrib 5.7.17, for osx10.12 (x86_64) using EditLine wrapper
• Django==1.9.13
• django-cms==3.4.3
• djangocms-googlemap==1.1.0
• Py2.7
• OS X 10.12.5 (16F73)

And, important: the version that is being migrated to the one mentioned above was djangocms-googlemap==0.4.0.

[Striv]

2¢, In my troubleshooting, I was trying to Migrate from an older version of the google plugin and I guess the model changed in between. I was able to fix this by directly removing all entries from the google maps table in the db, migrating, and adding them back in. I used: DELETE FROM djangocms_googlemap_googlemap WHERE 1==1; by executing: sqlite3 /PATH/TO/PROJECT/project.db < /PATH/TO/remove.sql Where remove.sql has that one line above in it. I suppose one could also unset the icon_id fields only and retain the data. In my case this was easiest and re-adding the information was not difficult. - Dan

…

On Jun 7, 2017, at 2:32 PM, Robert Pogorzelski ***@***.*** ***@***.***>> wrote: And, important: the version that is being migrated to the one mentioned above was djangocms-googlemap==0.4.0. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#63 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADjQSQs2LL3zLwbMfIrQyYwhhvtqmc2kks5sBuzSgaJpZM4NXMBu>.

While removing data to get things back normal is viable solution to some degree, in production system with customer's data one would expect migrations to work (being its very purpose of existence).

After having a look into migrations, the problem is probably in 0004_adapted_fields.py declaration of GoogleMapMarker. It has all fields defined except for icon. I'll give it a try.

Wrong, it's defined already in 0009_googlemapmarker_icon.py. Anyway, that's the place to look for.

[czpython]

Hello @Striv, @robpogorzelski
Thanks for following up.

The issue lies with the usage of the cms api in https://github.com/divio/djangocms-googlemap/blob/master/djangocms_googlemap/migrations/0005_create_nested_plugins.py

Because it uses the api, the model will be used directly which means that django will push the icon column defined in 0009, in this 0005 migration and so the database complains.

To fix this, we'll need to refactor that migration to use the cms plugin tree internals instead of the api.

I'm not sure which API we are talking about here, but the simple way to solve the issue is the changeset above. How about that? Works for me.

[czpython]

I'm referring to this call. We can't move the icon to a lower migration because that would be rewriting schema history which can bring other issues, like someone who upgrades from a version that higher than 0004 but lower than 0009 would be in trouble.

There is no good solution here, the only reasonable thing is to reduce the collateral damage. At least it would be possible to migrate up which currently is not the case. And if that's really the issue, then it's very unlikely that there is anyone being able to reach any newer migration than that.

Ok, now I see the point with API usage.

[czpython]

I'll try to push a fix by the end of the week

[czpython]

Hello @robpogorzelski,
Can you try the fix pushed in #65?
Thanks!

[rjbanna]

@Striv You can just add that column using simple SQL command
ALTER TABLE "table_name" ADD "icon_id" VARCHAR(50);