Domain matching query does not exist about modoboa-pfxadmin-migrate

[tonioo]

Hi, have you defined administrators in postfixadmin?

[tkald]

Yes, I have 2 admin users. For one admin (limited access) there was notice that password was set to superadmin password.
No info about superadmin was displayed though.

[tonioo]

What is your postfixadmin version?

[tonioo]

Both admins are limited? Or do you have a global admin?

[tkald]

One admin is superadmin, otherone is limited.
Postfixadmin 2.3.6.
And superadmin does have domain "ALL" in domain_admins.domain table

[tonioo]

But looks like the 'ALL' domain is missing from the domain table... Can you confirm that?

[tkald]

Yes, no "ALL" domain

[tonioo]

That's why the migration script fails... Do you know why this domain is missing? I thought it was mandatory...

[tkald]

I did some minor PA code modifications back in a day, can't recall anything related to ALL domain though :)
Ok, I can modify tables fo fit any requirements.

Maybe you can add to the docs expected PA database schema and other requirements. I did also have trouble with some accounts having "0000-00-00" modification date... as it is default value in my db schema.

[tonioo]

I can also try a fix to migrate super admins even if the ALL domain is not present. About dates, I don't think I can to something... The migration script is compatible with version 2.3 only.

[tkald]

Maybe you can set date now() if date is 0000-00-00 in PA database?

[tonioo]

@tkald could you try the 2 fixes I just pushed please?

[tkald]

Originally I also have null dates in PA domain table - it does fail with latest commits:

Traceback (most recent call last):
File "manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 353, in execute_from_command_line
utility.execute()
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute
output = self.handle(*args, **options)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 292, in handle
self._do_migration(options)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 284, in _do_migration
self._migrate_domain(pf_domain, options, creator)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 202, in _migrate_domain
self._migrate_mailboxes(newdom, options, creator)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 182, in _migrate_mailboxes
new_user.save(creator=creator, using=options["_to"])
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa/core/models.py", line 366, in save
super(User, self).save(*args, **kwargs)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 708, in save
force_update=force_update, update_fields=update_fields)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 736, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 820, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 859, in _do_insert
using=using, raw=raw)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/manager.py", line 122, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/query.py", line 1039, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1060, in execute_sql
cursor.execute(sql, params)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 117, in execute
six.reraise(utils.IntegrityError, utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2])
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 112, in execute
return self.cursor.execute(query, args)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.IntegrityError: (1048, "Column 'date_joined' cannot be null")

[tonioo]

Ok, just pushed a new fix. Can you try it?

…

On 23/01/2017 11:49, tkald wrote: Originally I also have null dates in PA domain table - it does fail with error: Traceback (most recent call last): File "manage.py", line 10, in execute_from_command_line(sys.argv) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/*init*.py", line 353, in execute_from_command_line utility.execute() File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/*init*.py", line 345, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv self.execute(*args, **cmd_options) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute output = self.handle(*args, **options) File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 292, in handle self._do_migration(options) File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 284, in _do_migration self._migrate_domain(pf_domain, options, creator) File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 202, in _migrate_domain self._migrate_mailboxes(newdom, options, creator) File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 182, in _migrate_mailboxes new_user.save(creator=creator, using=options["_to"]) File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa/core/models.py", line 366, in save super(User, self).save(*args, **kwargs) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 708, in save force_update=force_update, update_fields=update_fields) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 736, in save_base updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 820, in _save_table result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 859, in _do_insert using=using, raw=raw) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/manager.py", line 122, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/query.py", line 1039, in _insert return query.get_compiler(using=using).execute_sql(return_id) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1060, in execute_sql cursor.execute(sql, params) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 117, in execute six.reraise(utils.IntegrityError, utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2]) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 112, in execute return self.cursor.execute(query, args) File "/srv/modoboa/env/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute self.errorhandler(self, exc, value) File "/srv/modoboa/env/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorclass, errorvalue django.db.utils.IntegrityError: (1048, "Column 'date_joined' cannot be null") — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#10 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAgc7lggZz7qSksAA-JggxbQYsnGXgMvks5rVIWegaJpZM4LpyW9>.

[tkald]

Yes, works now! Thank you.

However I did get one more error stating that user name was too long:

django.db.utils.DataError: (1406, "Data too long for column 'last_name' at row 1")

I made name shorter in PA mailbox table and migration was success.

I do have next problem with my custom domain transports which are all gone in modoboa database.
I agree that you can't count with every custom installation but will it possibly work if I insert my transports directly into database? I did have some sympa transports there..

[tonioo]

Never tried to import transport entries from postfixadmin. Can you paste an example please?

[tkald]

With import, all those alternative transports became type: domain
Bigger problem is, there doesn't seems to be relevant data fields in modoboa database where to store this info.

[tonioo]

Actually, Modoboa does not provide a generic way to create transport entries... About your configuration, you only need to manually create a transport table for sympa mailing lists. Modoboa provides a vacation plugin, so vacation service in your example will be replaced. The virtual service is not used by modoboa and it is replaced by dovecot but the migration script already creates the needed data so you don't have to care about it.

[tkald]

Yeah, it's the sympa transport I'm concerned about.

Would be nice to have a way to define your own generic/custom transports if needed :)