django 1.9? about django-ttdb

[wilbuick]

I'll try have a look at this some time over the next few weeks.

[andreav]

Thank you!

[andreav]

Wow! Thank you!
I will try it as soon as possible!

[wilbuick]

Release 0.4 is available on pypi. The code is a bit more robust now and should hopefully work with future django versions as well.

One thing to note is that I did change the settings file implementation because django was being a bit picky about placing custom keys into the databases dict.

You now configure as follows using the TTDB tuple to contain a list of databases that you would like to enable the functionality.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'django_ttdb',
        'USER': 'postgres',
        'HOST': '127.0.0.1',
    }
}

TTDB = (
    'default',
)

[andreav]

Sorry but I cannot get it to work.
,
I created my template db with a command like this:

CREATE DATABASE trip2go_frontend TEMPLATE test_trip2go;

My settings.py is:

 DATABASES = {
    'default': {
        'ENGINE': 'django.contrib.gis.db.backends.postgis',
        'NAME': 'trip2go_frontend',
        'USER': 'trip2go',
        'PASSWORD': 'xxx',
        'HOST': '127.0.0.1',
        'PORT': '5432',
        'TEST': {
            'SERIALIZE': False,
        },
    }
}
TEST_RUNNER = 'ttdb.runner.TemplateDatabaseRunner'
TTDB = (
    'default',
)

When I run my tests

./manage.py test -v 2 --settings triptogo.settings.test-frontend

Creating test database for alias 'default' ('test_trip2go_frontend')...
Got an error creating the test database: source database "trip2go_frontend" is being accessed by other users
DETAIL:  There is 1 other session using the database.

Type 'yes' if you would like to try deleting the test database 'test_trip2go_frontend', or 'no' to cancel: 

But output from select * from pg_stat_activity ; says no one is connected

[wilbuick]

I have seen this problem before. In my case I was trying to run the test suite and at the same time I had the runserver running. I had to stop the runserver before running the test suite. After doing that it was working fine for me.

[andreav]

Thank you but this is not the case.
However If I break during test setup, I see from pg_stat_activity some code of mine executing a query.
This is for sure my fault, not ttdb app.

[andreav]

My factory boy classes are using factory.Iterator which makes queries at startup.
If I comment that code I can go on.
Database is succesfully created starting from template!! This is really great!

Just one note.
Test tearDown is very slow.
This seems due to the fact that Database 'SERIALIZE': False parameter seems to be ignored.

If I KeyboardInterrupt the test during teardown, this is the stack trace.

  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/test/runner.py", line 494, in run_suite
    ).run(suite)
  File "/usr/lib/python2.7/unittest/runner.py", line 151, in run
    test(result)
  File "/usr/lib/python2.7/unittest/suite.py", line 70, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python2.7/unittest/suite.py", line 113, in run
    self._tearDownPreviousClass(None, result)
  File "/usr/lib/python2.7/unittest/suite.py", line 241, in _tearDownPreviousClass
    tearDownClass()
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/ttdb/testcases.py", line 72, in tearDownClass
    reload_template_database(cls.template_database)
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/ttdb/utils.py", line 33, in reload_template_database
    connection.creation.create_test_db(verbosity=0)
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/ttdb/runner.py", line 22, in create_test_db
    self._old_create_test_db(*args, **kwargs)
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/db/backends/base/creation.py", line 78, in create_test_db
    self.connection._test_serialized_contents = self.serialize_db_to_string()
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/db/backends/base/creation.py", line 122, in serialize_db_to_string
    serializers.serialize("json", get_objects(), indent=None, stream=out)
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/core/serializers/__init__.py", line 129, in serialize
    s.serialize(queryset, **options)
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/core/serializers/base.py", line 95, in serialize
    self.handle_m2m_field(obj, field)
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/core/serializers/python.py", line 77, in handle_m2m_field
    for related in getattr(obj, field.name).iterator()]
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/db/models/query.py", line 52, in __iter__
    results = compiler.execute_sql()
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 837, in execute_sql
    sql, params = self.as_sql()
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 389, in as_sql
    where, w_params = self.compile(self.where) if self.where is not None else ("", [])
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 366, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/db/models/sql/where.py", line 79, in as_sql
    sql, params = compiler.compile(child)
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 366, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/db/models/lookups.py", line 157, in as_sql
    lhs_sql, params = self.process_lhs(compiler, connection)
  File "/home/main/Workspace/trip2go.proj/local/lib/python2.7/site-packages/django/db/models/lookups.py", line 153, in process_lhs
    lhs_sql = connection.ops.lookup_cast(self.lookup_name, field_internal_type) % lhs_sql
  File "/usr/lib/python2.7/unittest/signals.py", line 36, in __call__
    self.default_handler(signum, frame)
KeyboardInterrupt

You can see

serializers.serialize("json", get_objects(), indent=None, stream=out)

Thank you!

[wilbuick]

It shouldn't be being ignored. It should be being passed through correctly.

I think the problem that you are having is elsewhere.

There are three different test cases that you can use

TemplateDBTestCase - Does not reload the template db after each test
TemplateDBTransactionTestCase - Reloads the template db after each test
TemplateDBLiveServerTestCase - Reloads the template db after each test

Using the first class will be the fastest. The other two will be very slow because they drop and reload the test database using the template after each test.

However this behavior can be overridden.

class MyTestCase(TemplateDBTransactionTestCase):
    reload_after_test = False #disable recreating test db after each test

class MyTestCase(TemplateDBLiveServerTestCase):
    reload_after_test = False #disable recreating test db after each test

I recommend setting reload_after_test to False all the time unless your test case is making the database dirty and would cause subsequent tests to fail.

[andreav]

I'm using LiveServerTestCase class with @use_template_database decorator.

I noted than when test starts, I see invoking ttdb.runner.create_test_db with parameters (this is respecting settings DATABASE configuration):

self = <django.db.backends.postgresql.creation.DatabaseCreation object at 0x7f14802f6d50>
args = ()
kwargs = {'autoclobber': False, 'verbosity': 2, 'serialize': False, 'keepdb': False}

but when the same code is reached at tearDownClass() time, the same method is called with these parameters (note only verbosity param is passed)

self = <django.db.backends.postgresql.creation.DatabaseCreation object at 0x7f14802f6d50>
args = ()
kwargs = {'verbosity': 0}

If now I set kwargs = {'autoclobber': False, 'verbosity': 2, 'serialize': False, 'keepdb': False}
tearDown finishes immedeately.

[wilbuick]

Yes, you're quite right. I can see the place in the code that causes the problem. I'll have a look and see if I can sort it out sometime before christmas.

[wilbuick]

The serialize option is being handled correctly now. Have included tests for this new functionality. Please see the release 0.5 for this functionality.

[andreav]

I can confirm rel 0.5 works as expected!

Thank you very much for this work!