indietyp / django-automated-logging Goto Github PK

I am getting this info logged after my static requests:

Environment for request couldn't be determined. Request was not recorded.

for example:

02-28 11:44:33.741     INFO | automated_logging.signals.request.request_finished_signal:45   | Environment for request couldn't be determined. Request was not recorded.

This is not a problem in production, because static files are served separately, but it is a little confusing a noisy during development. Is there some error in my configuration? Or is there a way to exclude the static files from the logger?

I am just logging to the console, not to the using the database backend.

In case something else is stomping on an internal object, here is my middleware:

[
            "django.middleware.security.SecurityMiddleware",
            "whitenoise.middleware.WhiteNoiseMiddleware",
            "django.contrib.sessions.middleware.SessionMiddleware",
            "django.middleware.common.CommonMiddleware",
            "django.middleware.csrf.CsrfViewMiddleware",
            "django.contrib.auth.middleware.AuthenticationMiddleware",            
            "django.contrib.messages.middleware.MessageMiddleware",
            "django_plotly_dash.middleware.BaseMiddleware",
            "django_plotly_dash.middleware.ExternalRedirectionMiddleware",
            "django.middleware.clickjacking.XFrameOptionsMiddleware",
#          
# Some custom middleware that only affects the user model for impersonation
#
            "automated_logging.middleware.AutomatedLoggingMiddleware",
            "debug_toolbar.middleware.DebugToolbarMiddleware",
            "threadlocals.middleware.ThreadLocalMiddleware",
],

Otherwise thanks for a great project.

Thanks for this amazing module.
I have two questions:

1. How to know and record which user to do the operation, because I need to log the user's operation.

2. I could see you create several tables in the databse, if you could further explain their meaning and relation that would be really helpful.

Thanks again!

When I run python manage.py migrate automated_logging I am getting errors that models are already registered.

I was testing this app with django 2.1.8 and got this error when trying to check the admin:

File "/var/virtualenv/project/lib/python3.6/site-packages/automated_logging/admin.py", line 19, in get_actions
    del actions["delete_selected"]
KeyError: 'delete_selected'

An easy fix would be:

    def get_actions(self, request):
        actions = super(ReadOnlyAdminMixin, self).get_actions(request)
        actions.pop('delete_selected', None)
        return actions

Unable to use Admin site when decorator is added

Watching for file changes with StatReloader
Exception in thread django-main-thread:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/d_drive/Projects/Avantrio/Projects/venv/amat/lib/python3.8/site-packages/django/utils/autoreload.py", line 53, in wrapper
fn(*args, **kwargs)
File "/d_drive/Projects/Avantrio/Projects/venv/amat/lib/python3.8/site-packages/django/core/management/commands/runserver.py", line 110, in inner_run
autoreload.raise_last_exception()
File "/d_drive/Projects/Avantrio/Projects/venv/amat/lib/python3.8/site-packages/django/utils/autoreload.py", line 76, in raise_last_exception
raise _exception[1]
File "/d_drive/Projects/Avantrio/Projects/venv/amat/lib/python3.8/site-packages/django/core/management/init.py", line 357, in execute
autoreload.check_errors(django.setup)()
File "/d_drive/Projects/Avantrio/Projects/venv/amat/lib/python3.8/site-packages/django/utils/autoreload.py", line 53, in wrapper
fn(*args, **kwargs)
File "/d_drive/Projects/Avantrio/Projects/venv/amat/lib/python3.8/site-packages/django/init.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/d_drive/Projects/Avantrio/Projects/venv/amat/lib/python3.8/site-packages/django/apps/registry.py", line 122, in populate
app_config.ready()
File "/d_drive/Projects/Avantrio/Projects/venv/amat/lib/python3.8/site-packages/django/contrib/admin/apps.py", line 24, in ready
self.module.autodiscover()
File "/d_drive/Projects/Avantrio/Projects/venv/amat/lib/python3.8/site-packages/django/contrib/admin/init.py", line 24, in autodiscover
autodiscover_modules('admin', register_to=site)
File "/d_drive/Projects/Avantrio/Projects/venv/amat/lib/python3.8/site-packages/django/utils/module_loading.py", line 47, in autodiscover_modules
import_module('%s.%s' % (app_config.name, module_to_search))
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/d_drive/Projects/Avantrio/Projects/advantis-amat/backend/apps/users/admin.py", line 5, in
admin.site.register(User, UserAdmin)
File "/d_drive/Projects/Avantrio/Projects/venv/amat/lib/python3.8/site-packages/django/contrib/admin/sites.py", line 106, in register
for model in model_or_iterable:
TypeError: 'function' object is not iterable

thank you for your app
but I have a problem
it seems that it doesn't save traceback logs after exceptions occurred or for 500 responses.
is there a way to set this?

Per the settings and looking at [1] I think I should be seeing ip addresses in the log file. I haven't been able to track down any mistakes in my config and am hoping you could take a look?

[1] https://github.com/indietyp/django-automated-logging/blob/master/automated_logging/signals/request.py#L75

Here's some environment info:

Log message:

2021-07-28 18:31:47,597: INFO: [POST] [204] admin at /api/auth/login/

Output from pprint(namedtuple2dict(default)):

{'globals': {'exclude': {'applications': ['plain:basehttp',
                                          'plain:admin',
                                          'plain:contenttypes',
                                          'plain:migrations',
                                          'glob:session*']}},
 'model': {'detailed_message': True,
           'exclude': {'applications': [],
                       'fields': [],
                       'models': [],
                       'unknown': False},
           'loglevel': 20,
           'max_age': None,
           'performance': False,
           'snapshot': False},
 'modules': ['unspecified', 'model', 'request'],
 'request': {'data': {'content_types': ['application/json'],
                      'enabled': [],
                      'ignore': [],
                      'mask': ['password'],
                      'query': False},
             'exclude': {'applications': [],
                         'methods': ['GET'],
                         'status': [200],
                         'unknown': False},
             'ip': True,
             'log_request_was_not_recorded': True,
             'loglevel': 20,
             'max_age': None},
 'unspecified': {'exclude': {'applications': [], 'files': [], 'unknown': False},
                 'loglevel': 20,
                 'max_age': None}}

Logging configuration:

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "simple": {"format": "%(name)s:%(levelname)s: %(message)s"},
        "dal": {"format": "%(asctime)s: %(levelname)s: %(message)s"}
    },
    "handlers": {
        "automated_logging": {
            "level": "INFO",
            "class": "logging.handlers.WatchedFileHandler",
            "filename": "/tmp/dal.log",
            "formatter": "dal",
        },
        "console": {
            "class": "logging.StreamHandler",
            "formatter": "simple",
        }
    },
    "loggers": {
        "": {
            # The root logger
            "handlers": ["console"],
            "level": "INFO",
        },
        "automated_logging": {
            "handlers": ["automated_logging"],
            "level": "INFO",
            "propagate": False,
        },
    },
}

From pip list:

Django                          2.2.24
django-automated-logging        6.1.1
django-ipware                   3.0.2

python 3.7
Docker Engine 19.03.8
postgres 12.2

Running migrations:
api            |   Applying automated_logging.0001_initial... OK
api            |   Applying automated_logging.0002_auto_20180215_1540...Traceback (most recent call last):
api            |   File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 86, in _execute
api            |     return self.cursor.execute(sql, params)
api            | psycopg2.errors.CannotCoerce: cannot cast type integer to uuid
api            | LINE 1: ...LUMN "application_id" TYPE uuid USING "application_id"::uuid

I tried adding DAL to my project. Some of the models using ArrayField, JSONField using postgres lib.
For such models the audit log is not saved and errors out with the following exception.

File "/xxxxx/automated_logging/signals/save.py", line 97, in pre_save_signal
    for k, v in old.items()
TypeError: unhashable type: 'list'