kumar303 / hawkrest Goto Github PK

From #38:

One complication is that properly testing this change to hawkrest means overhauling the tests to actually test the middleware end to end, like eg:
https://github.com/encode/django-rest-framework/blob/master/tests/test_middleware.py

(And ideally also overhauling the auth parts too, a la https://github.com/encode/django-rest-framework/blob/master/tests/test_authentication.py)

#20 will shortly add Django 1.9 to the Travis run, but marked as allowed to fail, since the tests are currently failing, eg:

https://travis-ci.org/kumar303/hawkrest/jobs/101117531

py27-django1.9-drf3.3 runtests: commands[0] | /home/travis/build/kumar303/hawkrest/.tox/py27-django1.9-drf3.3/bin/python runtests.py

nosetests --logging-clear-handlers --with-nicedots --verbosity=1

Traceback (most recent call last):
  File "runtests.py", line 19, in <module>
    main()
  File "runtests.py", line 15, in main
    failures = test_runner.run_tests(sys.argv[1:])
  File "/home/travis/build/kumar303/hawkrest/.tox/py27-django1.9-drf3.3/lib/python2.7/site-packages/django_nose/runner.py", line 403, in run_tests
    result = self.run_suite(nose_argv)
  File "/home/travis/build/kumar303/hawkrest/.tox/py27-django1.9-drf3.3/lib/python2.7/site-packages/django_nose/runner.py", line 329, in run_suite
    django.setup()
  File "/home/travis/build/kumar303/hawkrest/.tox/py27-django1.9-drf3.3/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/travis/build/kumar303/hawkrest/.tox/py27-django1.9-drf3.3/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/home/travis/build/kumar303/hawkrest/.tox/py27-django1.9-drf3.3/lib/python2.7/site-packages/django/apps/config.py", line 90, in create
    module = import_module(entry)
  File "/opt/python/2.7.9/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/travis/build/kumar303/hawkrest/hawkrest/__init__.py", line 17, in <module>
    from rest_framework.authentication import BaseAuthentication
  File "/home/travis/build/kumar303/hawkrest/.tox/py27-django1.9-drf3.3/lib/python2.7/site-packages/rest_framework/authentication.py", line 13, in <module>
    from rest_framework.authtoken.models import Token
  File "/home/travis/build/kumar303/hawkrest/.tox/py27-django1.9-drf3.3/lib/python2.7/site-packages/rest_framework/authtoken/models.py", line 16, in <module>
    class Token(models.Model):
  File "/home/travis/build/kumar303/hawkrest/.tox/py27-django1.9-drf3.3/lib/python2.7/site-packages/django/db/models/base.py", line 94, in __new__
    app_config = apps.get_containing_app_config(module)
  File "/home/travis/build/kumar303/hawkrest/.tox/py27-django1.9-drf3.3/lib/python2.7/site-packages/django/apps/registry.py", line 239, in get_containing_app_config
    self.check_apps_ready()
  File "/home/travis/build/kumar303/hawkrest/.tox/py27-django1.9-drf3.3/lib/python2.7/site-packages/django/apps/registry.py", line 124, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

This appears to be due to Django 1.9 now not allowing models to be imported in __init__.py, per:
https://docs.djangoproject.com/en/1.7/internals/deprecation/#deprecation-removed-in-1-9

All models will need to be defined inside an installed application or declare an explicit app_label. Furthermore, it won’t be possible to import them before their application is loaded. In particular, it won’t be possible to import models inside the root package of their application.

However due to a bug, the deprecation warning for this wasn't shown when using previous versions of Django, otherwise the failure mode would have been more clear:
https://code.djangoproject.com/ticket/25477

Whilst hawkrest doesn't have any models itself, it imports several rest_framework classes which do import some of the rest_framework's models.

Once support for Django 1.9 is added, the allow_failure lines in .travis.yml can be removed.

I have a branch locally for adding Django 1.10/1.11 to the testing matrix.

However I noticed that even the current testing matrix is currently failing, on fresh pushes of master to my fork. eg:

1. https://travis-ci.org/edmorley/hawkrest/jobs/193424762

ImportError: cannot import name setup

(presumably newer django-nose is not compatible with Django 1.6):

2. https://travis-ci.org/edmorley/hawkrest/jobs/193424766

AssertionError: Unexpected call: log.warning("access denied: MisComputedContentHash: Our hash blPLW7mrURX5LO7m2f7iW+3kXihtvlX0//IigunDeCs= (sha256) did not match theirs e8gjS2dr/QZv4GEJa8HHIsupVVrGiWA7YVZnVStNv3c=")

We should first pin these dependencies to make master green, before I make further changes to the testing matrix.

The last (green) Travis run on master was:
https://travis-ci.org/kumar303/hawkrest

Comparing the 'py27-django1.8-drf3.3' job to that from my run, shows the following package changes:

--- working.txt 2017-01-19 17:07:00.306042800 +0000
+++ broken.txt  2017-01-19 17:06:46.381880000 +0000
@@ -1,26 +1,28 @@
-alabaster==0.7.8
+alabaster==0.7.9
+args==0.1.0
 Babel==2.3.4
-Django==1.8.13
-django-nose==1.4.3
+clint==0.5.1
+Django==1.8.17
+django-nose==1.4.4
 djangorestframework==3.3.3
-docutils==0.12
+docutils==0.13.1
 funcsigs==1.0.2
 hawkrest==0.0.10
 imagesize==0.7.1
-Jinja2==2.8
+Jinja2==2.9.4
 MarkupSafe==0.23
 mock==2.0.0
-mohawk==0.3.2.1
+mohawk==0.3.4
 nose==1.3.7
 nosenicedots==0.5
 pbr==1.10.0
-pkginfo==1.3.2
+pkginfo==1.4.1
 Pygments==2.1.3
-pytz==2016.4
-requests==2.10.0
-requests-toolbelt==0.6.2
+pytz==2016.10
+requests==2.12.5
+requests-toolbelt==0.7.0
 six==1.10.0
 snowballstemmer==1.2.1
-Sphinx==1.4.2
+Sphinx==1.5.1
 sphinx-rtd-theme==0.1.9
-twine==1.6.5
+twine==1.8.1

Similar to:
http://mohawk.readthedocs.org/en/latest/usage.html#logging

It would be good to document that auth errors can be debugged by adding a logger channel hawkrest to the Django logging config, with level of WARNING, so it outputs the full exception content here:

Would it be possible to create a new release that includes #16 ?

Thanks :-)

As of version 0.0.6 if you have rest_framework.permissions.IsAuthenticated in your permission classes, you'll see the following traceback:

web_1 | Traceback (most recent call last):
web_1 |   File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
web_1 |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
web_1 |   File "/usr/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
web_1 |     return view_func(*args, **kwargs)
web_1 |   File "/usr/local/lib/python2.7/site-packages/django/views/generic/base.py", line 71, in view
web_1 |     return self.dispatch(request, *args, **kwargs)
web_1 |   File "/usr/local/lib/python2.7/site-packages/rest_framework/views.py", line 466, in dispatch
web_1 |     response = self.handle_exception(exc)
web_1 |   File "/usr/local/lib/python2.7/site-packages/rest_framework/views.py", line 454, in dispatch
web_1 |     self.initial(request, *args, **kwargs)
web_1 |   File "/usr/local/lib/python2.7/site-packages/rest_framework/views.py", line 377, in initial
web_1 |     self.check_permissions(request)
web_1 |   File "/usr/local/lib/python2.7/site-packages/rest_framework/views.py", line 318, in check_permissions
web_1 |     if not permission.has_permission(request, self):
web_1 |   File "/usr/local/lib/python2.7/site-packages/rest_framework/permissions.py", line 48, in has_permission
web_1 |     return request.user and request.user.is_authenticated()
web_1 | AttributeError: 'DummyUser' object has no attribute 'is_authenticated'

As Hawkrest doesn't make use of django's User model, there seems to be no way to implement API permissions, throttling etc. as they all expect the presence of a User object on request.user. So at the minute it looks like I need to subclass HawkAuthentication to look up and return a User object, which means copy and pasting most of the authenticate() method.

Is there a recommended way of handling population of request.user? Or am I missing something generally?

First, thanks for creating this Django package.

Second, the docs mentioned to include HAWK_CREDENTIALS in my settings file. Does the package support retrieving this dict from the database? My use-case is a long list of mobile devices authenticating to my API. The information for each device is kept in the DB and would be too many to be manually add in the settings file. A device would serve as a "user" who is allowed to used my API.

Thanks.

This evening Treeherder experienced thousands of HTTP 500s of form:

Traceback (most recent call last): 
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/base.py", line 131, in get_response 
    response = middleware_method(request, response) 
  File "/app/.heroku/python/lib/python2.7/site-packages/newrelic/hooks/framework_django.py", line 333, in wrapper 
    return wrapped(*args, **kwargs) 
  File "/app/.heroku/python/lib/python2.7/site-packages/hawkrest/middleware.py", line 22, in process_response 
    raise RuntimeError('Django did not handle an incoming ' 
RuntimeError: Django did not handle an incoming Hawk request properly 

(Using hawkrest 1.0.0, mohawk 0.3.4, djangorestframework 3.6.4, Django 1.11.6, Python 2.7.14)

This exception only occurs if the authentication backend didn't process the request, but middleware.py's process_response() did.

From code inspection, one way this might occur, if is the HTTP_AUTHORIZATION header contained the string Hawk with no space before subsequent characters, since the conditional for both pieces aren't consistent in their use of whitespace:
https://github.com/kumar303/hawkrest/blob/1.0.0/hawkrest/__init__.py#L74
https://github.com/kumar303/hawkrest/blob/1.0.0/hawkrest/middleware.py#L11

Whilst looking at the middleware it also seems like it could be simplified somewhat, with either an early return, or at least a consolidation of conditionals - worth doing given middleware is on the hot path.

putting
'hawkrest.middleware.HawkResponseMiddleware'
in middleware classes
causes this error
File "/usr/local/lib/python2.7/dist-packages/django/utils/autoreload.py", line 228, in wrapper fn(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/channels/management/commands/runserver.py", line 51, in inner_run http_consumer=self.get_consumer(*args, **options), File "/usr/local/lib/python2.7/dist-packages/channels/management/commands/runserver.py", line 157, in get_consumer return StaticFilesConsumer() File "/usr/local/lib/python2.7/dist-packages/channels/handler.py", line 347, in __init__ self.handler = self.handler_class() File "/usr/local/lib/python2.7/dist-packages/channels/staticfiles.py", line 18, in __init__ super(StaticFilesHandler, self).__init__() File "/usr/local/lib/python2.7/dist-packages/channels/handler.py", line 194, in __init__ self.load_middleware() File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 82, in load_middleware mw_instance = middleware(handler) TypeError: this constructor takes no arguments

All traces of this can be removed

It was undocumented and was only used by the apk-signer which is obsolete now anyway.

I guess this wasn't documented too well in DRF (or I'm just dumb 💡) but exception values are sent as a response to the client, resulting in things like:

<Response [401]>
{"detail":"access denied: MacMismatch: MACs do not match; ours: mbWCYE2x2BwEw3BHbtscUOVy0lgI9mO+Tj9oKRrvySs=; theirs: 5tqRSdX+ev+oumz2/+saKY3Xrgf8kmFDqAXzCn5tigg="}

This is a potential security problem because it might give the attacker enough clues to figure break the keys.

In #11 / #12 the exact mohawk exception was suppressed for security reasons.

This was a good change, however I think it may have now swung too far the other way, in that I've been trying to debug an issue a client was having submitting to us, where they just see:
Hawk authentication failed

And the actual error message (seen after we increased the logging level on our webheads), was:
[2015-10-22 09:01:16,007] WARNING [hawkrest:81] access denied: TokenExpired: token with UTC timestamp 1445530846 has expired; it was compared to 1445529676

If we'd have at least seen the exception type of TokenExpired in the response to the client, then we'd have known sooner that they just needed to update their clocks.

Could we include the exception name in the response? eg:
Hawk authentication failed: TokenExpired

Thanks :-)

Please could you provide a wheel archive on PyPI alongside the standard package? http://pythonwheels.com/

Many thanks :-)

Hi,

we make a DELETE action with the Django REST framework. And then there is no data returned. And then a message is '204 No content'. As there is no content, there is also no content-type meta data in the response.

So now the signing is broken at https://github.com/kumar303/hawkrest/blob/master/hawkrest/middleware.py#L33 and is causing errors when you want to make a DELETE action to the Django REST API.

You should check if the header is actually set. If not, leave it out, or give it some default value.

eg:
https://travis-ci.org/edmorley/hawkrest/jobs/193482208

FAIL: tests/test_authentication.py:TestHawkAuthenticatedUser.test_method_compliance
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/edmorley/hawkrest/tests/test_authentication.py", line 215, in test_method_compliance
    .format(name))
AssertionError: HawkAuthenticatedUser is missing method: clean

option_list
This is the list of optparse options which will be fed
into the command's OptionParser for parsing arguments.
Deprecated and will be removed in Django 1.10.
Use add_arguments instead.

I have been using hawkrest and found the management command tool handy, although it's broken for later versions of Django and Py3+. Mind if I open a PR to address the issues below?

1. Django 1.8+ management command API uses argparse instead of optparse, so using the tool results in an AttributeError (BaseCommand has no attribute 'option_list') being raised (at least for 1.11).
2. ImportError raised due to attempting to import nonexistent function lookup_credentials.
3. SyntaxError raised due to print not being used as a function as required in py3.
4. Finally, I've added several unit tests for the management command itself.

Some of these have been brought up in other issues, but I have a PR ready to address them if it's ok to contribute.

In Django 1.10, AbstractBaseUser has had its is_authenticated() and is_anonymous() methods converted to attributes.

As such, HawkAuthenticatedUser will need updating to match.

See:
https://code.djangoproject.com/ticket/25847
django/django@c1aec0f

Currently the newest versions tested against are Django 1.9, django-rest-framework 3.3 and Python 3.5.

However there are now newer releases available:

• Django 1.10 and 1.11a1
• django-rest-framework 3.4 and 3.5
• Python 3.6

I have a branch locally to add testing for these (Django 1.10's tests are failing, so they'll initially be marked as "allowed to fail"). Once #28 is merged, I'll open a PR.