semente / django-smuggler Goto Github PK

Django Smuggler is a pluggable application for Django Web Framework that helps you to import/export fixtures via the automatically-generated administration interface.

License: GNU Lesser General Public License v3.0

Python 94.85% Makefile 0.56% HTML 4.59%

admin backup data database django fixtures python

django-smuggler's People

Contributors

Stargazers

Watchers

django-smuggler's Issues

SQL support

Add support to load/dump SQL data.

SQL data can be used as backup, once is more trusted than XML and JSON.

Smuggler's settings documentation

Settings on smuggler.settings need be documented in README.rst.

Loading data form keyerror

This is using python 3.5 and Django 1.9

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/site-packages/django/core/handlers/base.py", line 149, in get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/lib/python3.5/site-packages/django/core/handlers/base.py", line 147, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.5/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/django/views/generic/base.py", line 88, in dispatch
    return handler(request, *args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/django/views/generic/edit.py", line 213, in get
    return self.render_to_response(self.get_context_data())
  File "/usr/local/lib/python3.5/site-packages/smuggler/views.py", line 87, in get_context_data
    adminform=self.get_admin_form(kwargs['form']),
KeyError: 'form'

Saving to fixtures that are under version control

Our workflow would be something like this:

1.PERSON A: with smuggler, create app_fixture.json on the server
2. git add app_fixture.json; git commit -a
3. in django, change app data
4. with smuggler, click 'export' and overwrite app_fixture.json
5. git commit -a; git push
6. PERSON B:git pull, to get the latest app_fixture.json
7. with smuggler, click 'import' and import app_fixture.json

Person A and Person B now have an easy way to share fixtures.

The main point is, sometimes you want to 'save as...' and choose a filename, sometimes you want to 'save' over a default filename. Mostly, we will want to save over a default filename.

Python 3 support

I think it's worth trying to be compatible with both Python 2.6+, and Python 3.3+.

Import raises 403 error

This happens when I go to /admin/import/ and try to import a previously exported file.

Here is the output:
Reason given for failure:

CSRF token missing or incorrect.

In general, this can occur when there is a genuine Cross Site Request Forgery, or when Django's CSRF mechanism has not been used correctly. For POST forms, you need to ensure:

* The view function uses RequestContext for the template, instead of Context.
* In the template, there is a {% csrf_token %} template tag inside each POST form that targets an internal URL.
* If you are not using CsrfViewMiddleware, then you must use csrf_protect on any views that use the csrf_token template tag, as well as those that accept the POST data.

Django 1.7 tox tests failing

output:

Ran 39 tests in 6.091s

OK
Destroying test database for alias 'default'...
py27-dj17 create: /Users/peterconerly/code/django-smuggler/.tox/py27-dj17
py27-dj17 installdeps: https://github.com/django/django/tarball/stable/1.7.x, freezegun
py27-dj17 inst: /Users/peterconerly/code/django-smuggler/.tox/dist/django-smuggler-0.5.0.zip
py27-dj17 runtests: PYTHONHASHSEED='1369738868'
py27-dj17 runtests: commands[0] | python tests/run_tests.py
Traceback (most recent call last):
  File "tests/run_tests.py", line 15, in <module>
    management.call_command('test', 'test_app')
  File "/Users/peterconerly/code/django-smuggler/.tox/py27-dj17/lib/python2.7/site-packages/django/core/management/__init__.py", line 115, in call_command
    return klass.execute(*args, **defaults)
  File "/Users/peterconerly/code/django-smuggler/.tox/py27-dj17/lib/python2.7/site-packages/django/core/management/commands/test.py", line 71, in execute
    super(Command, self).execute(*args, **options)
  File "/Users/peterconerly/code/django-smuggler/.tox/py27-dj17/lib/python2.7/site-packages/django/core/management/base.py", line 331, in execute
    translation.activate('en-us')
  File "/Users/peterconerly/code/django-smuggler/.tox/py27-dj17/lib/python2.7/site-packages/django/utils/translation/__init__.py", line 145, in activate
    return _trans.activate(language)
  File "/Users/peterconerly/code/django-smuggler/.tox/py27-dj17/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 225, in activate
    _active.value = translation(language)
  File "/Users/peterconerly/code/django-smuggler/.tox/py27-dj17/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 209, in translation
    default_translation = _fetch(settings.LANGUAGE_CODE)
  File "/Users/peterconerly/code/django-smuggler/.tox/py27-dj17/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 189, in _fetch
    "The translation infrastructure cannot be initialized before the "
django.core.exceptions.AppRegistryNotReady: The translation infrastructure cannot be initialized before the apps registry is ready. Check that you don't make non-lazy gettext calls at import time.
ERROR: InvocationError: '/Users/peterconerly/code/django-smuggler/.tox/py27-dj17/bin/python tests/run_tests.py'
py33-dj17 create: /Users/peterconerly/code/django-smuggler/.tox/py33-dj17
py33-dj17 installdeps: https://github.com/django/django/tarball/stable/1.7.x, freezegun
py33-dj17 inst: /Users/peterconerly/code/django-smuggler/.tox/dist/django-smuggler-0.5.0.zip
py33-dj17 runtests: PYTHONHASHSEED='1369738868'
py33-dj17 runtests: commands[0] | python tests/run_tests.py
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/.tox/py33-dj17/lib/python3.3/site-packages/django/utils/translation/trans_real.py", line 186, in _fetch
    app_configs = reversed(list(apps.get_app_configs()))
  File "/Users/peterconerly/code/django-smuggler/.tox/py33-dj17/lib/python3.3/site-packages/django/apps/registry.py", line 137, in get_app_configs
    self.check_apps_ready()
  File "/Users/peterconerly/code/django-smuggler/.tox/py33-dj17/lib/python3.3/site-packages/django/apps/registry.py", line 124, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tests/run_tests.py", line 15, in <module>
    management.call_command('test', 'test_app')
  File "/Users/peterconerly/code/django-smuggler/.tox/py33-dj17/lib/python3.3/site-packages/django/core/management/__init__.py", line 115, in call_command
    return klass.execute(*args, **defaults)
  File "/Users/peterconerly/code/django-smuggler/.tox/py33-dj17/lib/python3.3/site-packages/django/core/management/commands/test.py", line 71, in execute
    super(Command, self).execute(*args, **options)
  File "/Users/peterconerly/code/django-smuggler/.tox/py33-dj17/lib/python3.3/site-packages/django/core/management/base.py", line 331, in execute
    translation.activate('en-us')
  File "/Users/peterconerly/code/django-smuggler/.tox/py33-dj17/lib/python3.3/site-packages/django/utils/translation/__init__.py", line 145, in activate
    return _trans.activate(language)
  File "/Users/peterconerly/code/django-smuggler/.tox/py33-dj17/lib/python3.3/site-packages/django/utils/translation/trans_real.py", line 225, in activate
    _active.value = translation(language)
  File "/Users/peterconerly/code/django-smuggler/.tox/py33-dj17/lib/python3.3/site-packages/django/utils/translation/trans_real.py", line 209, in translation
    default_translation = _fetch(settings.LANGUAGE_CODE)
  File "/Users/peterconerly/code/django-smuggler/.tox/py33-dj17/lib/python3.3/site-packages/django/utils/translation/trans_real.py", line 189, in _fetch
    "The translation infrastructure cannot be initialized before the "
django.core.exceptions.AppRegistryNotReady: The translation infrastructure cannot be initialized before the apps registry is ready. Check that you don't make non-lazy gettext calls at import time.
ERROR: InvocationError: '/Users/peterconerly/code/django-smuggler/.tox/py33-dj17/bin/python tests/run_tests.py'
________________________________________________________________________________________ summary ________________________________________________________________________________________
  py26-dj14: commands succeeded
  py26-dj15: commands succeeded
  py27-dj16: commands succeeded
  py33-dj16: commands succeeded
ERROR:   py27-dj17: commands failed
ERROR:   py33-dj17: commands failed

I tried switching

from django.utils.translation import ugettext as _

in smuggler/forms.py and smuggler/views.py
to

from django.utils.translation import ugettext_lazy as _

But that hasn't fixed the issue.

TemplateDoesNotExist: smuggler/load_data_form.html

I am using 0.6.0 and although the /dump functionality works fine, I get a TemplateDoesNotExist: smuggler/load_data_form.html error when appending /load to an admin URL.

My urls.py is as follows:

from django.conf.urls import patterns, include, url
from django.contrib import admin
from django.conf import settings
from django.views.generic import TemplateView

from core.views import home

admin.autodiscover()

urlpatterns = patterns('',
    # Examples:
    # url(r'^$', 'tmr.views.home', name='home'),
    # url(r'^blog/', include('blog.urls')),

    url(r'^admin/', include('smuggler.urls')),
    url(r'^admin/', include(admin.site.urls)),
    url(r'^$', home, name='home'),
)

urlpatterns += patterns('',
    url('', include('social.apps.django_app.urls', namespace='social')),
)
if not settings.DEBUG:
    urlpatterns += patterns('',
        (r'^static/(?P<path>.*)$', 'django.views.static.serve', {'document_root': settings.PROJECT_ROOT}),
    )

I'm sure I'm overlooking something here, but I can't find what on earth it could be.

allow dev to set url for login page when not logged in.

Trying to access /admin/dump when admin is not logged in now dumps the user to /accounts/login/?next=/admin/dump/, which appears to be hard-wired into the app.

My app has its login page at /admin/login/, not /accounts/login/.

Can we have a setting that lets us change to login page

thanks for a useful app!

John

Release version 0.9

Now PR #58 is merged everything should be ready for a new release.

Could not parse the remainder: '-data' from 'load-data'. The syntax of 'url' changed in Django 1.5, see the docs.

Hello

The package is not totally compatible with django 1.5.1.
load-data for a url is not working
it works with load_data

I want contribute to the project for debugging of compatibility Django higher 1.5

Thanks for the response.

Gerard Toko

Optional / after urls.

This is incredibly minor, but having optional /es would be nice.

When I hit 127.0.0.1:8000/admin/dump it'll just give me a 404.

coveralls command is failing

coveralls command is failing and breaking travis-ci build. Based on the command error output I even have added the file https://github.com/semente/django-smuggler/blob/master/.coveralls.yml but with no success, build is still broken.

For reference: https://travis-ci.org/semente/django-smuggler/jobs/93181185

@jaap3, do you have any idea?

Fix broken tests

So this kinda sucks, tox breaks right now. Some of it is on simple stuff, like a dictionary that was ordered differently, but some of the errors seem to come from a change in FlatPage structure.

(smuggler)[peterconerly@django-smuggler]$ tox
GLOB sdist-make: /Users/peterconerly/code/django-smuggler/setup.py
py25-1.2 create: /Users/peterconerly/code/django-smuggler/.tox/py25-1.2
ERROR: UnsupportedInterpreter: python2.5 is not supported anymore, sorry
py25-1.3 create: /Users/peterconerly/code/django-smuggler/.tox/py25-1.3
ERROR: UnsupportedInterpreter: python2.5 is not supported anymore, sorry
py26-1.3 inst-nodeps: /Users/peterconerly/code/django-smuggler/.tox/dist/django-smuggler-0.4.1.zip
py26-1.3 runtests: PYTHONHASHSEED='3745116371'
py26-1.3 runtests: commands[0] | python tests/run_tests.py
Creating test database for alias 'default'...
FFF.
======================================================================
FAIL: test_serialize_exclude (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 33, in test_serialize_exclude
    self.assertEquals(out, '[ %s ]' % self.FLATPAGE_DUMP)
AssertionError: '[ { "fields": { "enable_comments": false, "registration_required": false, "template_name": "", "title": "test", "sites": [], "url": "/", "content": "" }, "model": "flatpages.flatpage", "pk": 1 } ]' != '[ { "pk": 1, "model": "flatpages.flatpage", "fields": { "registration_required": false, "title": "test", "url": "/", "template_name": "", "sites": [], "content": "", "enable_comments": false } } ]'

======================================================================
FAIL: test_serialize_include (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 39, in test_serialize_include
    self.assertEquals(out, '[ %s ]' % self.SITE_DUMP)
AssertionError: '[ { "fields": { "name": "example.com", "domain": "example.com" }, "model": "sites.site", "pk": 1 } ]' != '[ { "pk": 1, "model": "sites.site", "fields": { "domain": "example.com", "name": "example.com" } } ]'

======================================================================
FAIL: test_serialize_to_response (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 27, in test_serialize_to_response
    self.assertEquals(out, self.BASIC_DUMP)
AssertionError: '[ { "fields": { "name": "example.com", "domain": "example.com" }, "model": "sites.site", "pk": 1 }, { "fields": { "enable_comments": false, "registration_required": false, "template_name": "", "title": "test", "sites": [], "url": "/", "content": "" }, "model": "flatpages.flatpage", "pk": 1 } ]' != '[ { "pk": 1, "model": "sites.site", "fields": { "domain": "example.com", "name": "example.com" } }, { "pk": 1, "model": "flatpages.flatpage", "fields": { "registration_required": false, "title": "test", "url": "/", "template_name": "", "sites": [], "content": "", "enable_comments": false } } ]'

----------------------------------------------------------------------
Ran 4 tests in 0.007s

FAILED (failures=3)
Destroying test database for alias 'default'...
ERROR: InvocationError: '/Users/peterconerly/code/django-smuggler/.tox/py26-1.3/bin/python tests/run_tests.py'
py26-1.4 inst-nodeps: /Users/peterconerly/code/django-smuggler/.tox/dist/django-smuggler-0.4.1.zip
py26-1.4 runtests: PYTHONHASHSEED='3745116371'
py26-1.4 runtests: commands[0] | python tests/run_tests.py
Creating test database for alias 'default'...
FFF.
======================================================================
FAIL: test_serialize_exclude (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 33, in test_serialize_exclude
    self.assertEquals(out, '[ %s ]' % self.FLATPAGE_DUMP)
AssertionError: '[ { "fields": { "enable_comments": false, "registration_required": false, "template_name": "", "title": "test", "sites": [], "url": "/", "content": "" }, "model": "flatpages.flatpage", "pk": 1 } ]' != '[ { "pk": 1, "model": "flatpages.flatpage", "fields": { "registration_required": false, "title": "test", "url": "/", "template_name": "", "sites": [], "content": "", "enable_comments": false } } ]'

======================================================================
FAIL: test_serialize_include (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 39, in test_serialize_include
    self.assertEquals(out, '[ %s ]' % self.SITE_DUMP)
AssertionError: '[ { "fields": { "name": "example.com", "domain": "example.com" }, "model": "sites.site", "pk": 1 } ]' != '[ { "pk": 1, "model": "sites.site", "fields": { "domain": "example.com", "name": "example.com" } } ]'

======================================================================
FAIL: test_serialize_to_response (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 27, in test_serialize_to_response
    self.assertEquals(out, self.BASIC_DUMP)
AssertionError: '[ { "fields": { "name": "example.com", "domain": "example.com" }, "model": "sites.site", "pk": 1 }, { "fields": { "enable_comments": false, "registration_required": false, "template_name": "", "title": "test", "sites": [], "url": "/", "content": "" }, "model": "flatpages.flatpage", "pk": 1 } ]' != '[ { "pk": 1, "model": "sites.site", "fields": { "domain": "example.com", "name": "example.com" } }, { "pk": 1, "model": "flatpages.flatpage", "fields": { "registration_required": false, "title": "test", "url": "/", "template_name": "", "sites": [], "content": "", "enable_comments": false } } ]'

----------------------------------------------------------------------
Ran 4 tests in 0.006s

FAILED (failures=3)
Destroying test database for alias 'default'...
ERROR: InvocationError: '/Users/peterconerly/code/django-smuggler/.tox/py26-1.4/bin/python tests/run_tests.py'
py26-1.5 inst-nodeps: /Users/peterconerly/code/django-smuggler/.tox/dist/django-smuggler-0.4.1.zip
py26-1.5 runtests: PYTHONHASHSEED='3745116371'
py26-1.5 runtests: commands[0] | python tests/run_tests.py
Creating test database for alias 'default'...
FFF.
======================================================================
FAIL: test_serialize_exclude (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 33, in test_serialize_exclude
    self.assertEquals(out, '[ %s ]' % self.FLATPAGE_DUMP)
AssertionError: '[ { "fields": { "enable_comments": false, "registration_required": false, "template_name": "", "title": "test", "sites": [], "url": "/", "content": "" }, "model": "flatpages.flatpage", "pk": 1 } ]' != '[ { "pk": 1, "model": "flatpages.flatpage", "fields": { "registration_required": false, "title": "test", "url": "/", "template_name": "", "sites": [], "content": "", "enable_comments": false } } ]'

======================================================================
FAIL: test_serialize_include (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 39, in test_serialize_include
    self.assertEquals(out, '[ %s ]' % self.SITE_DUMP)
AssertionError: '[ { "fields": { "name": "example.com", "domain": "example.com" }, "model": "sites.site", "pk": 1 } ]' != '[ { "pk": 1, "model": "sites.site", "fields": { "domain": "example.com", "name": "example.com" } } ]'

======================================================================
FAIL: test_serialize_to_response (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 27, in test_serialize_to_response
    self.assertEquals(out, self.BASIC_DUMP)
AssertionError: '[ { "fields": { "name": "example.com", "domain": "example.com" }, "model": "sites.site", "pk": 1 }, { "fields": { "enable_comments": false, "registration_required": false, "template_name": "", "title": "test", "sites": [], "url": "/", "content": "" }, "model": "flatpages.flatpage", "pk": 1 } ]' != '[ { "pk": 1, "model": "sites.site", "fields": { "domain": "example.com", "name": "example.com" } }, { "pk": 1, "model": "flatpages.flatpage", "fields": { "registration_required": false, "title": "test", "url": "/", "template_name": "", "sites": [], "content": "", "enable_comments": false } } ]'

----------------------------------------------------------------------
Ran 4 tests in 0.008s

FAILED (failures=3)
Destroying test database for alias 'default'...
ERROR: InvocationError: '/Users/peterconerly/code/django-smuggler/.tox/py26-1.5/bin/python tests/run_tests.py'
py27-1.3 inst-nodeps: /Users/peterconerly/code/django-smuggler/.tox/dist/django-smuggler-0.4.1.zip
py27-1.3 runtests: PYTHONHASHSEED='3745116371'
py27-1.3 runtests: commands[0] | python tests/run_tests.py
Creating test database for alias 'default'...
FFF.
======================================================================
FAIL: test_serialize_exclude (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 33, in test_serialize_exclude
    self.assertEquals(out, '[ %s ]' % self.FLATPAGE_DUMP)
AssertionError: '[ { "fields": { "enable_comments": false, "registration_required": false, "template_name": "", "title": "test", "sites": [], "url": "/", "content": "" }, "model": "flatpages.flatpage", "pk": 1 } ]' != '[ { "pk": 1, "model": "flatpages.flatpage", "fields": { "registration_required": false, "title": "test", "url": "/", "template_name": "", "sites": [], "content": "", "enable_comments": false } } ]'

======================================================================
FAIL: test_serialize_include (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 39, in test_serialize_include
    self.assertEquals(out, '[ %s ]' % self.SITE_DUMP)
AssertionError: '[ { "fields": { "name": "example.com", "domain": "example.com" }, "model": "sites.site", "pk": 1 } ]' != '[ { "pk": 1, "model": "sites.site", "fields": { "domain": "example.com", "name": "example.com" } } ]'

======================================================================
FAIL: test_serialize_to_response (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 27, in test_serialize_to_response
    self.assertEquals(out, self.BASIC_DUMP)
AssertionError: '[ { "fields": { "name": "example.com", "domain": "example.com" }, "model": "sites.site", "pk": 1 }, { "fields": { "enable_comments": false, "registration_required": false, "template_name": "", "title": "test", "sites": [], "url": "/", "content": "" }, "model": "flatpages.flatpage", "pk": 1 } ]' != '[ { "pk": 1, "model": "sites.site", "fields": { "domain": "example.com", "name": "example.com" } }, { "pk": 1, "model": "flatpages.flatpage", "fields": { "registration_required": false, "title": "test", "url": "/", "template_name": "", "sites": [], "content": "", "enable_comments": false } } ]'

----------------------------------------------------------------------
Ran 4 tests in 0.006s

FAILED (failures=3)
Destroying test database for alias 'default'...
ERROR: InvocationError: '/Users/peterconerly/code/django-smuggler/.tox/py27-1.3/bin/python tests/run_tests.py'
py27-1.4 inst-nodeps: /Users/peterconerly/code/django-smuggler/.tox/dist/django-smuggler-0.4.1.zip
py27-1.4 runtests: PYTHONHASHSEED='3745116371'
py27-1.4 runtests: commands[0] | python tests/run_tests.py
Creating test database for alias 'default'...
FFF.
======================================================================
FAIL: test_serialize_exclude (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 33, in test_serialize_exclude
    self.assertEquals(out, '[ %s ]' % self.FLATPAGE_DUMP)
AssertionError: '[ { "fields": { "enable_comments": false, "registration_required": false, "template_name": "", "title": "test", "sites": [], "url": "/", "content": "" }, "model": "flatpages.flatpage", "pk": 1 } ]' != '[ { "pk": 1, "model": "flatpages.flatpage", "fields": { "registration_required": false, "title": "test", "url": "/", "template_name": "", "sites": [], "content": "", "enable_comments": false } } ]'

======================================================================
FAIL: test_serialize_include (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 39, in test_serialize_include
    self.assertEquals(out, '[ %s ]' % self.SITE_DUMP)
AssertionError: '[ { "fields": { "name": "example.com", "domain": "example.com" }, "model": "sites.site", "pk": 1 } ]' != '[ { "pk": 1, "model": "sites.site", "fields": { "domain": "example.com", "name": "example.com" } } ]'

======================================================================
FAIL: test_serialize_to_response (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 27, in test_serialize_to_response
    self.assertEquals(out, self.BASIC_DUMP)
AssertionError: '[ { "fields": { "name": "example.com", "domain": "example.com" }, "model": "sites.site", "pk": 1 }, { "fields": { "enable_comments": false, "registration_required": false, "template_name": "", "title": "test", "sites": [], "url": "/", "content": "" }, "model": "flatpages.flatpage", "pk": 1 } ]' != '[ { "pk": 1, "model": "sites.site", "fields": { "domain": "example.com", "name": "example.com" } }, { "pk": 1, "model": "flatpages.flatpage", "fields": { "registration_required": false, "title": "test", "url": "/", "template_name": "", "sites": [], "content": "", "enable_comments": false } } ]'

----------------------------------------------------------------------
Ran 4 tests in 0.007s

FAILED (failures=3)
Destroying test database for alias 'default'...
ERROR: InvocationError: '/Users/peterconerly/code/django-smuggler/.tox/py27-1.4/bin/python tests/run_tests.py'
py27-1.5 inst-nodeps: /Users/peterconerly/code/django-smuggler/.tox/dist/django-smuggler-0.4.1.zip
py27-1.5 runtests: PYTHONHASHSEED='3745116371'
py27-1.5 runtests: commands[0] | python tests/run_tests.py
Creating test database for alias 'default'...
FFF.
======================================================================
FAIL: test_serialize_exclude (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 33, in test_serialize_exclude
    self.assertEquals(out, '[ %s ]' % self.FLATPAGE_DUMP)
AssertionError: '[ { "fields": { "enable_comments": false, "registration_required": false, "template_name": "", "title": "test", "sites": [], "url": "/", "content": "" }, "model": "flatpages.flatpage", "pk": 1 } ]' != '[ { "pk": 1, "model": "flatpages.flatpage", "fields": { "registration_required": false, "title": "test", "url": "/", "template_name": "", "sites": [], "content": "", "enable_comments": false } } ]'

======================================================================
FAIL: test_serialize_include (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 39, in test_serialize_include
    self.assertEquals(out, '[ %s ]' % self.SITE_DUMP)
AssertionError: '[ { "fields": { "name": "example.com", "domain": "example.com" }, "model": "sites.site", "pk": 1 } ]' != '[ { "pk": 1, "model": "sites.site", "fields": { "domain": "example.com", "name": "example.com" } } ]'

======================================================================
FAIL: test_serialize_to_response (test_app.tests.BasicDumpTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/peterconerly/code/django-smuggler/tests/test_app/tests.py", line 27, in test_serialize_to_response
    self.assertEquals(out, self.BASIC_DUMP)
AssertionError: '[ { "fields": { "name": "example.com", "domain": "example.com" }, "model": "sites.site", "pk": 1 }, { "fields": { "enable_comments": false, "registration_required": false, "template_name": "", "title": "test", "sites": [], "url": "/", "content": "" }, "model": "flatpages.flatpage", "pk": 1 } ]' != '[ { "pk": 1, "model": "sites.site", "fields": { "domain": "example.com", "name": "example.com" } }, { "pk": 1, "model": "flatpages.flatpage", "fields": { "registration_required": false, "title": "test", "url": "/", "template_name": "", "sites": [], "content": "", "enable_comments": false } } ]'

----------------------------------------------------------------------
Ran 4 tests in 0.008s

FAILED (failures=3)
Destroying test database for alias 'default'...
ERROR: InvocationError: '/Users/peterconerly/code/django-smuggler/.tox/py27-1.5/bin/python tests/run_tests.py'
________________________________________________________________ summary _________________________________________________________________
ERROR:   py25-1.2: UnsupportedInterpreter: python2.5 is not supported anymore, sorry
ERROR:   py25-1.3: UnsupportedInterpreter: python2.5 is not supported anymore, sorry
ERROR:   py26-1.3: commands failed
ERROR:   py26-1.4: commands failed
ERROR:   py26-1.5: commands failed
ERROR:   py27-1.3: commands failed
ERROR:   py27-1.4: commands failed
ERROR:   py27-1.5: commands failed

Create a json file

One of the main problems is the loading time for preparing the file. Meanwhile, I suggest an option where we can enable to use of multiprocessing, in order to return the request back and then create a .json file in the file repository, and then it can be efficient. Currently, for my use case, I need to set timeout to a high value, which is not a good solution.

Smuggler has only template hack for change_list.html

Create templates to add export/import buttons on app indexes, for exports data from a app, and admin index, for exports data from whole project.

Add documentation for signals

Add comments to signals on signals.py and a note on README.rst.

Add management commands

Version 1.0.0 not available on PyPi

In the README and GitHub Releases there is a version 1.0.0 with support for Django 3.0, but it's not listed in PyPi:
https://pypi.org/project/django-smuggler/#history

When attempting to install it via pipenv I get the following error message:

ERROR: ERROR: Could not find a version that matches django-smuggler>=1.0.0
Tried: 0.2, 0.3, 0.4, 0.4.1, 0.5.0, 0.5.0, 0.6.0, 0.6.0, 0.6.1, 0.6.1, 0.7.0, 0.7.0, 0.8.0, 0.8.0, 0.9.0, 0.9.0, 0.9.1, 0.9.1

Release a new version with Django 1.5 compat

Django 1.5 has been out for a while, smuggler is now compatible with this version so a new release is in order.

Most important changes are the ones done for issue #22 and #23

Allow exporting of selected model objects with Admin actions

Create an admin action (http://docs.djangoproject.com/en/1.1/ref/contrib/admin/actions/#ref-contrib-admin-actions) to allow one or more model objects to be selected and exported.

Add tests

CSRF token

Forms break without CRSF token tags(see http://docs.djangoproject.com/en/dev/ref/contrib/csrf/ )

I created a fix on my fork http://github.com/brettgoulder/django-smuggler/commit/1cf5102765b3438781ca4dd995ddd8171a8ce6ec

Setup Travis and Coveralls for this project

Get the test suite up to snuff (and close issue #9, #32) then try to set up https://travis-ci.org and https://coveralls.io

/admin/dump/ gives KeyError: 'no_color' on Django 1.10

I am using Django 1.10 and I suspect it is part of the problem.

Accessing /admin/load works fine.

Accessing /admin/dump gives the following traceback:

Environment:

Request Method: GET
Request URL: http://127.0.0.1:8000/admin/dump/

Django Version: 1.10
Python Version: 3.5.2
Installed Applications:
('django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'myapp',
 'smuggler')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'django.middleware.security.SecurityMiddleware')



Traceback:

File "/usr/lib/python3.5/site-packages/django/core/handlers/exception.py" in inner
  39.             response = get_response(request)

File "/usr/lib/python3.5/site-packages/django/core/handlers/base.py" in _legacy_get_response
  249.             response = self._get_response(request)

File "/usr/lib/python3.5/site-packages/django/core/handlers/base.py" in _get_response
  187.                 response = self.process_exception_by_middleware(e, request)

File "/usr/lib/python3.5/site-packages/django/core/handlers/base.py" in _get_response
  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/usr/lib/python3.5/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
  23.                 return view_func(request, *args, **kwargs)

File "/usr/lib/python3.5/site-packages/smuggler/views.py" in dump_data
  68.                             exclude=settings.SMUGGLER_EXCLUDE_LIST)

File "/usr/lib/python3.5/site-packages/smuggler/views.py" in dump_to_response
  41.         response = serialize_to_response(app_label, exclude)

File "/usr/lib/python3.5/site-packages/smuggler/utils.py" in serialize_to_response
  40.         'use_natural_primary_keys': True

File "/usr/lib/python3.5/site-packages/django/core/management/base.py" in execute
  325.         if options['no_color']:

Exception Type: KeyError at /admin/dump/
Exception Value: 'no_color'

/admin/load will give the same error after uploading a file.

Dump filenames without spaces

Filenames for /app/model/dump end up as app-model-_2021-02-05T12 25 46.775165.yaml for me. It would be simpler to use these filenames didnt contain spaces ( ), or even :. Also the _ before the year, i.e. -_2021 is a bit odd. i.e. ISO 8601 without the colons would be ideal, especially in UTC given the file could be moved across timezones, and thus the stamp should be terminated with Z.

Support for dumping with natural keys

Hi there, thanks for making a supremely useful application.

I was wondering if there was a way to dump data with natural keys, such that I could easily dump my Users model and load it into a clean install without getting 'ContentType matching query does not exist' type errors (i.e. errors for data I don't really need).

Thanks for reading.

'url' is not a valid tag or filter in tag library 'future'

This is the error I get in django admin running on python 3.5 django 1.9

'url' is not a valid tag or filter in tag library 'future'
Request Method: GET
Request URL:    http://localhost:8000/admin/app/currency/
Django Version: 1.9.2
Exception Type: TemplateSyntaxError
Exception Value:    
'url' is not a valid tag or filter in tag library 'future'
Exception Location: /usr/local/lib/python3.5/site-packages/django/template/defaulttags.py in load_from_library, line 1157
Python Executable:  /usr/local/opt/python3/bin/python3.5
Python Version: 3.5.1
Python Path:    
['/Users/jerry/Desktop/Programming/Solanum',
 '/Users/jerry/Desktop/Programming/Solanum/pip_modules',
 '/Users/jerry/Desktop/Programming/Solanum',
 '/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python35.zip',
 '/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5',
 '/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/plat-darwin',
 '/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload',
 '/usr/local/lib/python3.5/site-packages']

Django 1.9+ jquery.min.js staticfile

Got an error when using a custom staticfile handler, because it could not find
/admin/js/jquery.min.js in the location.

According to douglasmiranda/django-admin-bootstrap#88 the jquery file is from 1.9 onwards in /admin/js/vendor/jquery/jquery.min.js

Simple workaround for me now was to copy the file over to old place as well.

But this should probably be fixed from the package itself so that form media is loaded correctly depending on version.

Update screenshots on README

Update screenshots to reflect recent changes.

Drop support for Django 1.3

Django 1.3 is no longer supported by the Django Foundation itself. Django 1.4 is an LTS release (https://docs.djangoproject.com/en/dev/internals/release-process/#long-term-support-lts-releases), so it's worth supporting.

README is incorrect (urls)

Just installed, pretty cool stuff! Anyway, the README says to use /dump/ and /load/. The urls.py that comes with smuggler uses /export/ and /import/.

Incompatible with virtual models

This may be a 'wontfix', but could be worth documenting, ...

I am using https://github.com/idlesign/django-siteprefs which creates virtual models in apps named app.preferences.

I am also using django-haystackbrowser

smuggler 'dump' endpoint fails because of these models.

An exception occurred while dumping data: Unable to serialize database: no such table: haystackbrowser_haystackresults

I can put them SMUGGLER_EXCLUDE_LIST, and it may be that is the most appropriate approach.

However if there is no table, there is nothing to dump, and the dump process could skip past these, perhaps leaving a note, maybe in the fixture?

The only other time this will happen is when a real table for a normal model hasnt been created yet, perhaps because a migration hasnt been run. In which case, the site has bigger problems, and even then, the admin probably would like smuggler to do the dump despite the problem, as they may be trying to rescue a site. Again in that case the admin could put the broken table into SMUGGLER_EXCLUDE_LIST, but that could require a few more complicated steps to edit that file if the site is deployed on the cloud or a remote locked down host.

load_data_form.html is incompatible with Django 1.5

The 'admin_media_prefix' template tag has been deprecated and removed. It also seems that the url tag is still "old-style".

PermissionError on Windows when using the 'load' action with uploaded files

I am using Windows python and Django. Tried django-smuggler and found this issue:

Succeed on 'dump', and created file: 2018-04-27T11_05_53.741567.json

on a new database, tried '/admin/load', it did display a page and asking for the file. clicked the file that just saved, then clicked 'load' button. It shows an error:

PermissionError at /admin/load/
[WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\Ken\AppData\Local\Temp\repair_2018-04-27T15_44_57.883458_fz9nmkw9.json'

I notice that the file name has been changed during the process, but i don't know why? or did I do anything wrong?

Changes in aa105b3 needs documentation

Load view not working

I'm running Django 2.2.5 on Python 3.7, and I've noticed that, when trying to open the 'load' view in DEBUG=False, it produces a Server error (500) view; it does, however, function correctly in DEBUG=True. Please, consider looking into this. Thanks a lot.

My settings.py can be found in here and my urls.py is here. If you need any more information please reply to the issue.

Getting debug information in my dump

see:

[    �[32;1m[sql]�[0m �[1mSELECT ...
      FROM "frontend_userprofile"
      ORDER BY "frontend_userprofile"."id" ASC�[0m
    �[32;1m[sql]�[0m �[1mSELECT ...
      FROM "auth_user"
      WHERE "auth_user"."id" = 2�[0m
{
  "pk": 1, 
  "model": "frontend.userprofile", 
  "fields": {
    "acknowledged_warnings": "0", 
    "royale_expiration": null, 
    "json_properties": null, 
    "subscription_method": null, 
    "email_verified": false, 
    "paypal_customer_id": null, 
    "braintree_customer_id": null, 
    "user": [
      "[email protected]"
    ], 
    "subscription_id": null, 
    "email_optin": false, 
    "screenname": "peter"
  }
}

Is there an option I should be changing?

Have an /admin/smuggler/ page

So. I have a many-to-many relationship, and I obsfucate the through model from the django admin, b/c my coworkers don't need to see it.

I'd like to be able to replicate our production data into our staging / QA / dev environments. So there are three specific models I need to download the data from.

It'd be awesome if there were a general smuggler page. Something that would just list all apps and models, and you could check boxes for what you want to dump, and then a "dump" button.

/auth
/djcelery
/app
  /unimportant model
  /unimportant model 2
  X article
  X group
  X group-article
/app2
  /foo
  /bar

@semente what are your thoughts about a feature like this?

Is there some easy way I can overload a dump() call to give me 3 specific models, so I can at least hack a solution for myself?

Add admin.logentry in the list of excluded models

Support to import/export user uploaded data (from the default media storage)

Figure out why coverage is not 100% on Django 1.9

We're not hitting these lines in Django 1.9 https://coveralls.io/builds/4274269/source?filename=smuggler%2Fviews.py#L158

Need to find out why and fix it.

ProgrammingError when an app has a model with managed=false

I have an app where the database tables don't actually exist:

class Report(models.Model):
    class Meta:
        managed = False

When accessing smuggler via the url:
http://localhost:8000/admin/export/

I get:

ProgrammingError: (1146, "Table 'myapp.reporting_report' doesn't exist")

which is obviously true but stops me using Smuggler for the rest of my apps.

Keep uploaded files as alternative choices to import

List files from SMUGGLER_FIXTURE_DIR to import via admin interface.

Models with foreign key to auth.user instances can't be loaded correctly

When dumping the auth data the user ids/primary keys are not dumped, so when loading the file again, other models having ForeignKeys to User instances can't be loaded properly.
When loading a file with dumped auth data, the users are imported correctly but get new ids/primary keys.

Can not dump contenttypes app

JSON cannot be deleted on Windows because of PermissionError

Hi.

Reason: Load of JSON files fail when trying to delete the JSON file.

Expected: Loading successful. No error. Temporary file deleted.

Reproducible: On Windows 10 (not tested in other OS), with any JSON.

Below, the traces.

Request Method:	POST
http://localhost:5000/admin/load/
2.2.1
PermissionError
[WinError 32] El proceso no tiene acceso al archivo porque está siendo utilizado por otro proceso: 'C:\Users\Ivan\AppData\Local\Temp\auth_34g73qrx.json'
C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\lib\site-packages\smuggler\views.py in form_valid, line 138
C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\python.exe
3.7.1
['C:\ATAVIA\src', 'C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\python37.zip', 'C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\DLLs', 'C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\lib', 'C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia', 'C:\Users\Ivan\AppData\Roaming\Python\Python37\site-packages', 'C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\lib\site-packages', 'C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\lib\site-packages\odf', 'C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\lib\site-packages\odf', 'C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\lib\site-packages\odf', 'C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\lib\site-packages\odf', 'C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\lib\site-packages\odf', 'C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\lib\site-packages\odf', 'C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\lib\site-packages\odf']
Lun, 2 Dic 2019 08:49:23 +0100

=================================================

C:\Users\Ivan\AppData\Local\conda\conda\envs\atavia\lib\site-packages\smuggler\views.py in form_valid

                os.unlink(tmp_file)

Variable	Value
class	<class 'smuggler.views.LoadDataView'>
destination_path	'C:\Users\Ivan\AppData\Local\Temp\auth_34g73qrx.json'
file_name	'auth.json'
fixtures	['C:\Users\Ivan\AppData\Local\Temp\auth_34g73qrx.json']
form
picked_files	[]
prefix	'auth'
self	<smuggler.views.LoadDataView object at 0x0000028927536CC0>
store	False
suffix	'.json'
tmp_file	'C:\Users\Ivan\AppData\Local\Temp\auth_34g73qrx.json'
tmp_fixtures	['C:\Users\Ivan\AppData\Local\Temp\auth_34g73qrx.json']
upload	<InMemoryUploadedFile: auth.json (application/json)>
uploads	[<InMemoryUploadedFile: auth.json (application/json)>]

==================================================

Please excuse the Spanish localization.

Regards

Permission denied in windows, solution explained

Hello,
In some windows, this bug happens:
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process

The way to fix it:
https://stackoverflow.com/questions/27215462/permissionerror-winerror-32-the-process-cannot-access-the-file-because-it-is

I'd appreciate devs if they can apply this to the project.

semente / django-smuggler Goto Github PK

django-smuggler's People

Contributors

Stargazers

Watchers

Forkers

django-smuggler's Issues

Below, the traces.

Recommend Projects

Recommend Topics

Recommend Org

Jobs