jazzband / django-ddp Goto Github PK

django-ddp always responds with SockJS-style framing, regardless of the URL used to connect to the websocket

For example: a["{\"server_id\":\"0\"}"]

Using /websocket and /sockjs/websocket should return raw, unframed data while /sockjs/999/foo/websocket should use the SockJS framing.

I don't know how to fix this in a python2 and python3 compatible way sorry. Your pypy entry does suggest python3 is supported https://pypi.python.org/pypi/django-ddp/0.19.1

I had to make this change for python3.5

--- a/dddp/views.py
+++ b/dddp/views.py
@@ -26,7 +26,7 @@ def dict_merge(lft, rgt):
if not isinstance(rgt, dict):
return rgt
result = deepcopy(lft)

• for key, val in rgt.iteritems():

• for key, val in rgt.items():
  if key in result and isinstance(result[key], dict):
  result[key] = dict_merge(result[key], val)
  else:

Traceback (most recent call last):
File "/var/www/html/py35_ev/lib/python3.5/site-packages/gevent/pywsgi.py", line 936, in handle_one_response
self.run_application()
File "/var/www/html/py35_ev/lib/python3.5/site-packages/geventwebsocket/handler.py", line 88, in run_application
return super(WebSocketHandler, self).run_application()
File "/var/www/html/py35_ev/lib/python3.5/site-packages/gevent/pywsgi.py", line 910, in run_application
self.process_result()
File "/var/www/html/py35_ev/lib/python3.5/site-packages/gevent/pywsgi.py", line 896, in process_result
self.write(data)
File "/var/www/html/py35_ev/lib/python3.5/site-packages/gevent/pywsgi.py", line 742, in write
self._write_with_headers(data)
File "/var/www/html/py35_ev/lib/python3.5/site-packages/gevent/pywsgi.py", line 764, in _write_with_headers
self._write(data)
File "/var/www/html/py35_ev/lib/python3.5/site-packages/gevent/pywsgi.py", line 724, in _write
self._sendall(data)
File "/var/www/html/py35_ev/lib/python3.5/site-packages/gevent/pywsgi.py", line 702, in _sendall
self.socket.sendall(data)
File "/var/www/html/py35_ev/lib/python3.5/site-packages/gevent/_socket3.py", line 408, in sendall
data_memory = _get_memory(data)
File "/var/www/html/py35_ev/lib/python3.5/site-packages/gevent/_socket3.py", line 35, in _get_memory
mv = memoryview(data)
TypeError: memoryview: a bytes-like object is required, not 'str'
Thu Nov 3 11:47:04 2016 {'REMOTE_ADDR': '127.0.0.1', 'REMOTE_PORT': '56104', 'HTTP_HOST': '127.0.0.1:8000', (hidden keys: 23)} failed with TypeError

Edit: Project not abandoned, mentoring available!

Hai,

This project looks abandoned? I have real site using django-dddp

Great project !Would love to see an small example (possibly in the documentation) for handling server side scripting :)

Accounts.logoutOtherClients is not implemented.
Suggest that we simple delete other dddp.models.Connection instances for the same user, dispatch async notifications for the PostgresWorker threads to kill the relevant WebSocket client threads.

For example calling an unknown method returns the following:
a["{\"msg\": \"error\", \"error\": \"Unknown method: foo\"}"]

Meteor won't pass errors like the above through to the error argument of the callback (example below).

Django.call('foo', function(error, response) {
  if (error)
    console.log(error);
});

IMHO, it should be possible to check out the project on a brand new machine, and run tox.

I believe there is some requirement for meteor to be installed?

Thanks for your time on this project. I'm getting back into working with it and it's really great! I had some questions about relations and what data is serialized and sent down to Meteor. I mentioned this before, but apparently related models through foreign keys only serialize and pass down the ID of the object, not access to the other fields.

For example, I have a model that is related to a User. In Meteor I would like to display the name of that user when referencing the object. I tried to add it through annotating with an F expression on the Django queryset, but that doesn't get serialized either. I do see in the api.py serialize method where you check for F expressions, but mine never match the isinstance(val, ExpressionNode) check as val is seen as a unicode string (in the case of the username). I don't know if this is a bug in that method or a user error on my end constructing the expression. The value is there, but it never gets serialized.

My workaround was that I just added in a bit at the end of your serialize method that finds all methods marked as a @Property and then serialize and add them to the payload that's returned. This works great and I like to use properties on my models anyway, so this works fine for me. ( I can put up a pull request for this if you think it's useful )

But I'm surely missing something on how to do this otherwise -- so what is the recommended way to work with additional fields on related objects in this way?

With dddp 0.19.0, I started seeing this error bubble up to the client:

autoupdate subscription failed: M…r.m…e.errorClass {error: 404, reason: "Subscription not found", details: undefined, message: "Subscription not found [404]", errorType: "Meteor.Error"} - referring to meteor_autoupdate_clientVersions

I found that the addition of the id_ in the websocket send bit of do_sub method (line 703 of api.py) is causing the error to now show. Prior to 0.19.0, this publication error occurred in dddp, but never showed up on the client, it seems.

Everything appears to work just fine, but the error will show up many times on the client as Meteor retries the subscription. So, with that, does this mean that dddp needs a subscription endpoint for it, or should the do_sub function silently ignore the error? I believe that meteor_autoupdate_clientVersions is a client-only local collection and isn't on the server anyway.

Hi there,

Firstly, thanks for working on this project.

I'm wondering if anyone is using this in production, and/or if there's any reasons to not consider it for production use at this point?

Also on that topic, does anyone have any example code on how this would be deployed in a production environment using gunicorn? I'm guessing it's more complicated than simply pointing gunicorn at the dddp command.

Many thanks,
Richard

Configure CI service (eg: travis-ci) to run test suite automatically.

(From https://gitter.im/commoncode/django-ddp?at=56551f41bcd12a10513a0be3)

Given the Assessment model:

class Assessment(models.Model):
    invited = models.ManyToManyField('auth.user')

class Assessment(Collection):

    """Assessment DDP Collection."""

    model = assessment.Assessment
    user_rel = [
       'invited',
    ]

Just curious to see what a project structure where you mesh these 2 would look like. This would be project paradise for me if i can get this working :)

As Django Channels is slated to be a part of django in an upcoming release, would you consider adding support for it in django-ddp?

The Meteor docs section on Passwords includes some methods that Django DDP doesn't support. Figure out which methods should be supported and implement them.

Hello!

I'm using Sqlite Db in my project and starting using django-ddp:

  File "/Users/user/Desktop/HuskyJam/common_venv/lib/python3.4/site-packages/dddp/apps.py", line 28, in ready
    alias, conf['backend'],
KeyError: 'backend'

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}

 def ready(self):
        """Initialisation for django-ddp (setup lookups and signal handlers)."""
        if not settings.DATABASES:
            raise ImproperlyConfigured('No databases configured.')
        for (alias, conf) in settings.DATABASES.items():
            if conf['ENGINE'] != 'django.db.backends.postgresql_psycopg2':
                raise ImproperlyConfigured(
                    '%r uses %r: django-ddp only works with PostgreSQL.' % (
                        alias, conf['backend'],
                    )
                )
        self.api = autodiscover()
        self.api.ready()

I am trying to integrate a DDP Client into my Django app on the client side as explained here.

I believe from the post, that everything should work fine, except that I want to add an authentication layer for my publish and subscribe calls. Only a "check" for userLoggedIn to determine if the client is authorized to subscribe is enough.

However, I plan to have the user login done through Django and I want to modify the authentication check in Meteor to use the Django authentication system.

Any idea how I can use the DDP "request" from the DDP client to determine if the user has already been authenticated by Django or not.

I am new to DDP and Django (!) and wondering if there are any session type things in the DDP "request" that I can use on the server side to determine if the user has been authenticated by Django already.

I hope I am able to explain it clearly ... Please help!

Hi guys,

I am looking into meteor right now and the only thing keeping me away is the nosql mongodb database.
This module looks like the solution!!!

Can you tell me about your experience using it?
Are you using it in production?
Are you happy with it?
Are there any problems that might occur?
Do all meteor-functions work?

I'd really love to hear some feedback.

Kind regards
Mikey

The code around https://github.com/commoncode/django-ddp/blob/develop/dddp/api.py#L49 for supporting the array_agg PostgreSQL aggregate function works for Django 1.7 and 1.8, but it's ugly and needs to be cleaned up.

I am a newbie in angular and i have recently try django.
I have searched a complete tutorial from zero for using django ddp which seems to be the greatest web technology that i ever touch, but there is a crucial lack of it avoiding me to use django ddp.
I think it could be such a good idea to make a simple one with every state of the set up and a demo, Or if a such tutorial exist to link it in the documentation.

Anyway great job and still try harding on django ddp

Please collect ideas and links under this issue here. I've heard of Facebook's fork of MySQL with listen/notify support. Any links? PostgreSQL docs say that listen/notify are not standard SQL. Is there a way to implement live in a database independent manner.

Hi,

I'm sorry for this question I am not yet an expert to both django and meteorjs. I am trying to use this django-ddp technology but I am a little stuck on "Start the Django DDP service" on the Example Usage

I created a virtualenv,
I created a project named tutorial,
I followed the example usage instructions from the start until,
Every time I tried to run this command (DJANGO_SETTINGS_MODULE=tutorial.settings dddp) in shell I always get a response of "ImportError: No module named tutorial.settings"

P.S.: I even tried to package the project dir but still no luck.

I really like to practice and use django-ddp ever since I watched your presentation in pycon AU at youtube.

Hi please check the images as a result of attempt to install django-ddp

I'm getting the below error

Environment:

Request Method: GET
Request URL: http://localhost:8000/

Django Version: 1.8.5
Python Version: 3.4.3
Installed Applications:
('django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'dddp',
'todo.django_todos')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware')

Traceback:
File "/home/kj/.virtualenvs/django-ddp/lib/python3.4/site-packages/django/core/handlers/base.py" in get_response

1.                 response = wrapped_callback(request, _callback_args, *_callback_kwargs)
   

   File "/home/kj/.virtualenvs/django-ddp/lib/python3.4/site-packages/django/views/generic/base.py" in view

2.         self = cls(**initkwargs)
   

   File "/home/kj/.virtualenvs/django-ddp/lib/python3.4/site-packages/dddp/views.py" in init

3.     self.meteor_settings = reduce(
   

Exception Type: NameError at /
Exception Value: name 'reduce' is not defined

this is the error i'm getting - using python 2.7

ProgrammingError at /admin/django_todos/task/

relation "django_todos_task" does not exist LINE 1: SELECT COUNT(*) AS "__count" FROM "django_todos_task"

Exception Value: relation "django_todos_task" does not exist LINE 1: SELECT COUNT(*) AS "__count" FROM "django_todos_task"

When running dddp for https://github.com/tysonclugg/django-ddp-example (which has no wsgi.py file) the following error occurs:

$ dddp --settings=todo.todo.settings
Traceback (most recent call last):
  File "/home/tclugg/.virtualenvs/django-ddp/bin/dddp", line 9, in <module>
    load_entry_point('django-ddp==0.18.0', 'console_scripts', 'dddp')()
  File "/home/tclugg/projects/django-ddp/dddp/main.py", line 338, in main
    verbosity=namespace.verbosity,
  File "/home/tclugg/projects/django-ddp/dddp/main.py", line 268, in serve
    launcher = DDPLauncher(debug=verbosity == 3, verbosity=verbosity)
  File "/home/tclugg/projects/django-ddp/dddp/main.py", line 114, in __init__
    self.wsgi_app = import_string(self.wsgi_name)
  File "/home/tclugg/.virtualenvs/django-ddp/lib/python3.4/site-packages/django/utils/module_loading.py", line 26, in import_string
    module = import_module(module_path)
  File "/home/tclugg/.virtualenvs/django-ddp/lib/python3.4/importlib/__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2224, in _find_and_load_unlocked
ImportError: No module named 'todo.wsgi'

This is a question, no bug.

1. I'm just asking wether django-ddp is considered stable enough to be used for a medical application, where data loss/failure is a more-or-less no-go, and should be maintainable for many years.
   How stable is django-ddp considered to be for that?
   Would you recommend it for using in a "data loss critical" environment?

I know that this question is a bit broad, and answers maybe opinion-based, but I want to ask it nevertheless.

2. Is django-ddp on the server and python-ddp on the client a stable option? Any experiences here?
   Thank you very much.

I suppose someone had to ask for it. But it would be great to have python 3 support now that gevent==1.1b2 has it. (well you did mention it in your recent pycon Oz talk)

Documentation needs to be written and published covering all aspects of using Django DDP.

Hello,

Got an issue when i'm using gevent 1.1b6 (python3.4 and django 1.8)

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/kj/.virtualenvs/freelance/lib/python3.4/site-packages/django/core/management/__init__.py", line 351, in execute_from_command_line
    utility.execute()
  File "/home/kj/.virtualenvs/freelance/lib/python3.4/site-packages/django/core/management/__init__.py", line 325, in execute
    django.setup()
  File "/home/kj/.virtualenvs/freelance/lib/python3.4/site-packages/django/__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/kj/.virtualenvs/freelance/lib/python3.4/site-packages/django/apps/registry.py", line 115, in populate
    app_config.ready()
  File "/home/kj/.virtualenvs/freelance/lib/python3.4/site-packages/dddp/apps.py", line 31, in ready
    self.api = autodiscover()
  File "/home/kj/.virtualenvs/freelance/lib/python3.4/site-packages/dddp/__init__.py", line 137, in autodiscover
    autodiscover_modules('ddp', register_to=API)
  File "/home/kj/.virtualenvs/freelance/lib/python3.4/site-packages/django/utils/module_loading.py", line 74, in autodiscover_modules
    import_module('%s.%s' % (app_config.name, module_to_search))
  File "/home/kj/.virtualenvs/freelance/lib/python3.4/importlib/__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1471, in exec_module
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "/home/kj/.virtualenvs/freelance/lib/python3.4/site-packages/dddp/accounts/ddp.py", line 568, in <module>
    API.register([Users, LoginServiceConfiguration, LoggedInUser, Auth])
  File "/home/kj/.virtualenvs/freelance/lib/python3.4/site-packages/dddp/api.py", line 792, in register
    self._registry[api.api_path_prefix],
dddp.AlreadyRegistered: API with prefix '/' is already registered to <dddp.ddp.Logs object at 0x7fd5d1a2d4a8>

Documentation around serializing unsupported types like decimals in a Collection would be great.

There's been a surge of interest recently with lots of discussions both online and offline - but none in any central location.

Create an online presence for Django DDP developers to connect with.

Some parts from readme are not clear for me, especially paths and urls when i want integrate django-ddp as the primary DDP. It will be nice if i can look into demo project, it can be an example with bookstore from readme

Write tests and document how developers can run them.

Hi,

Getting the below error when attempting to migrate the database.

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/django/core/management/__init__.py", line 351, in execute_from_command_line
    utility.execute()
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/django/core/management/__init__.py", line 325, in execute
    django.setup()
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/django/__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/django/apps/config.py", line 86, in create
    module = import_module(entry)
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/importlib/__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1471, in exec_module
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/dddp/__init__.py", line 5, in <module>
    from gevent.local import local
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/gevent/__init__.py", line 41, in <module>
    from gevent.hub import get_hub, iwait, wait
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/gevent/hub.py", line 289
    except Exception, ex:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/django/core/management/__init__.py", line 351, in execute_from_command_line
    utility.execute()
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/django/core/management/__init__.py", line 325, in execute
    django.setup()
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/django/__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/django/apps/registry.py", line 115, in populate
    app_config.ready()
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/dddp/apps.py", line 31, in ready
    self.api = autodiscover()
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/dddp/__init__.py", line 141, in autodiscover
    autodiscover_modules('ddp', register_to=API)
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/django/utils/module_loading.py", line 74, in autodiscover_modules
    import_module('%s.%s' % (app_config.name, module_to_search))
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/importlib/__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1471, in exec_module
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "/home/kj/Desktop/django-meteor/todo/django_todos/ddp.py", line 16, in <module>
    Tasks,
  File "/home/kj/.virtualenvs/meteor-django/lib/python3.4/site-packages/dddp/api.py", line 787, in register
    if api.api_path_prefix in self._registry:
AttributeError: 'Task1' object has no attribute 'api_path_prefix'

The default serializer fails to return a value for the foo field of the Bar model in following:

from django.db import models

class Foo(models.Model):
    name = models.CharField(max_length=10)

class Bar(models.Model):
    foo = models.ForeignKey('Foo', db_column='foo')

I'm running into issues when defining get_queries() on publications with additional/optional arguments.

This is the example from the docs of passing an argument to get_queries:

class BooksByAuthorEmail(Publication):
    def get_queries(self, author_email):
        return [
            models.Author.objects.filter(
                email=author_email,
            ),
            models.Book.objects.filter(
                author__email=author_email,
            ),
        ]

class BooksByAuthorEmail(Publication):
    def get_queries(self, env, author_email):
        return [
            models.Author.objects.filter(
                email=author_email,
            ),
            models.Book.objects.filter(
                author__email=author_email,
            ),
        ]

Traceback (most recent call last):
  File "/home/richard/.pyenv/versions/playversity/src/dddp/dddp/websocket.py", line 276, in process_ddp
    self.dispatch(msg, data)
  File "/home/richard/.pyenv/versions/playversity/local/lib/python2.7/site-packages/django/utils/decorators.py", line 184, in inner
    return func(*args, **kwargs)
  File "/home/richard/.pyenv/versions/playversity/src/dddp/dddp/websocket.py", line 351, in dispatch
    handler(**kwargs)
  File "/home/richard/.pyenv/versions/playversity/src/dddp/dddp/websocket.py", line 476, in recv_method
    self.api.method(method, params, id_)
  File "/home/richard/.pyenv/versions/playversity/src/dddp/dddp/api.py", line 768, in method
    result = handler(*params)
  File "/home/richard/.pyenv/versions/playversity/src/dddp/dddp/accounts/ddp.py", line 469, in login
    return self.login_with_resume_token(params)
  File "/home/richard/.pyenv/versions/playversity/src/dddp/dddp/accounts/ddp.py", line 514, in login_with_resume_token
    self.do_login(user)
  File "/home/richard/.pyenv/versions/playversity/src/dddp/dddp/accounts/ddp.py", line 441, in do_login
    self.update_subs(user.pk)
  File "/home/richard/.pyenv/versions/playversity/src/dddp/dddp/accounts/ddp.py", line 267, in update_subs
    in API.sub_unique_objects(sub, params, pub)
  File "/home/richard/.pyenv/versions/playversity/src/dddp/dddp/api.py", line 629, in sub_unique_objects
    in pub.user_queries(sub.user, *params)
  File "/home/richard/.pyenv/versions/playversity/src/dddp/dddp/api.py", line 558, in user_queries
    return get_queries(*params)
TypeError: get_queries() takes at least 2 arguments (1 given)
TypeError('get_queries() takes at least 2 arguments (1 given)',)
> /home/richard/.pyenv/versions/playversity/src/dddp/dddp/api.py(558)user_queries()
    557         try:
--> 558             return get_queries(*params)
    559         finally:

Support for using PyPy would be nice.