AppRegistryNotReady: Apps aren't loaded yet about django-celery-monitor

[jezdez]

Okay, I'll take another look at how to solve this in the camera module, which may as simple as inlining the import to prevent the import time side effect from happening -- on Monday though :)

[jezdez]

Thanks all, gonna do a quick release..

[jezdez]

How did you launch the camera?
How does your celery.py look like?

[psychok7]

@jezdez

i launched camera with: celery -A proj events -l info --camera django_celery_monitor.camera.Camera --frequency=2.0 like mentioned in the docs. I have this in a docker-compose service.

my celery.py

from __future__ import absolute_import

import os
import celery

from django.conf import settings

# set the default Django settings module for the 'celery' program.
if os.environ.get('DJANGO_SETTINGS_MODULE') is None:
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings.production')


print('DJANGO_SETTINGS_MODULE: ', os.environ.get('DJANGO_SETTINGS_MODULE'))


# Based  on http://stackoverflow.com/a/27620486/977622 for sending sentry errors
class Celery(celery.Celery):

    def on_configure(self):
        if hasattr(settings, 'RAVEN_CONFIG') and settings.RAVEN_CONFIG['dsn']:
            import raven
            from raven.contrib.celery import (register_signal,
                                              register_logger_signal)

            client = raven.Client(settings.RAVEN_CONFIG['dsn'])
            register_logger_signal(client)
            register_signal(client)

app = Celery('proj')

# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)


@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

[jezdez]

Okay, so the issue is that the Django app registry isn't loaded yet when the events command tries to import the Camera class. That's expected behavior at least from Django's side. This is basically a case of how to use Django in "standalone" scripts.

In Celery django.setup() is for some reason only called before launching the workers and not when running other commands. Which means, all you have to do in your celery.py is add a call to django.setup() after setting the DJANGO_SETTINGS_MODULE environment variable.

I just checked the Celery code for extension points where we could do this automatically in django-celery-monitor but I'm afraid I couldn't find one in the code path that the events command takes, so that the only alternative would be the hooks in the celery.Celery class to load fixups. Which I don't think makes sense for this app to provide, it's just too much effort compared to "add the call to django.setup()" to your celery.py.

[psychok7]

@jezdez if i add django.setup() the errors do stop, but i start getting import errors regarding other apps that i have installed (django-compressor). I also don't think celerybeat works properly with that as i only see in my logs Writing entries.... If i remove django.setup() everything starts working again except for this app.

[jezdez]

@psychok7 Could you please try out the code in the app-registry branch (or PR #3)?

[psychok7]

@jezdez i still have the same issue django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.. Installed commit f42b15e . I did not add django.setup() to my celery.py because of the celerybeat issue.

[psychok7]

@arnemorten do you have everything working including celerybeat?

[jezdez]

@psychok7 Could you paste the traceback again please?

[psychok7]

@jezdez my traceback:

celerycam_1   | Traceback (most recent call last):
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/kombu/async/timer.py", line 166, in apply_entry
celerycam_1   |     entry()
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/kombu/async/timer.py", line 68, in __call__
celerycam_1   |     return self.fun(*self.args, **self.kwargs)
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/kombu/async/timer.py", line 127, in _reschedules
celerycam_1   |     return fun(*args, **kwargs)
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/celery/events/snapshot.py", line 70, in capture
celerycam_1   |     self.state.freeze_while(self.shutter, clear_after=self.clear_after)
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/celery/events/state.py", line 444, in freeze_while
celerycam_1   |     return fun(*args, **kwargs)
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/celery/events/snapshot.py", line 67, in shutter
celerycam_1   |     self.on_shutter(self.state)
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/django_celery_monitor/camera.py", line 137, in on_shutter
celerycam_1   |     self.handle_worker(worker)
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/django_celery_monitor/camera.py", line 73, in handle_worker
celerycam_1   |     obj, _ = self.WorkerState.objects.update_or_create(
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/django_celery_monitor/camera.py", line 40, in WorkerState
celerycam_1   |     return symbol_by_name('django_celery_monitor.models.WorkerState')
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/kombu/utils/imports.py", line 56, in symbol_by_name
celerycam_1   |     module = imp(module_name, package=package, **kwargs)
celerycam_1   |   File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
celerycam_1   |     return _bootstrap._gcd_import(name[level:], package, level)
celerycam_1   |   File "<frozen importlib._bootstrap>", line 986, in _gcd_import
celerycam_1   |   File "<frozen importlib._bootstrap>", line 969, in _find_and_load
celerycam_1   |   File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
celerycam_1   |   File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
celerycam_1   |   File "<frozen importlib._bootstrap_external>", line 665, in exec_module
celerycam_1   |   File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/django_celery_monitor/models.py", line 21, in <module>
celerycam_1   |     class WorkerState(models.Model):
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/django/db/models/base.py", line 110, in __new__
celerycam_1   |     app_config = apps.get_containing_app_config(module)
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/django/apps/registry.py", line 247, in get_containing_app_config
celerycam_1   |     self.check_apps_ready()
celerycam_1   |   File "/usr/local/lib/python3.5/dist-packages/django/apps/registry.py", line 125, in check_apps_ready
celerycam_1   |     raise AppRegistryNotReady("Apps aren't loaded yet.")
celerycam_1   | django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

[arnemorten]

@psychok7 Spoke too soon. I had fixed it by just importing django and running django.setup() in camera.py. I thought I was running the patch but i wasn't.

The new behaviour with the patch is that I can start the event listener fine, but when a worker launches I get the same Apps aren't loaded yet.

[jezdez]

@psychok7 @arnemorten Okay, another try with 136cacc?

[arnemorten]

@jezdez No errors and I'm getting Tasks populated in the database.

[jezdez]

@arnemorten Huzzah!

[psychok7]

@jezdez everything seems to be working fine now with your latest commit

[psychok7]

@jezdez thanks for the effort