yunojuno / django-onfido

Django app for integration with Onfido

Django Onfido

Django app for integration with the Onfido API (v3)

The current version supports Django 3.1+/4.0+ and Python 3.7+.

For v2 API please check version prior to 1.0

Background

Onfido is an online identity verification service. It provides API access to a range of tests (identity, right to work, criminal history, credit report). It is assumed that you are only interested in this project because you are already aware of what Onfido does, and so I won't repeat it here. If you want to find out more, head over to their website.

If you are using Onfido, and you are using Django, then this project can be used to manage Onfido checks against your existing Django users. It handles the API interactions, as well as providing the callback webhooks required to support live status updates.

Installation

The project is available through PyPI as django-onfido:

$ pip install django-onfido

And the main package itself is just onfido:

>>> from onfido import api, models, views, urls, admin, signals, helpers, decorators

Usage

The main use case is as follows:

1. Create an Onfido Applicant from your Django user:

>>> from django.contrib.auth import get_user_model
>>> from onfido.helpers import create_applicant
>>> user = get_user_model().objects.last()  # any old one will do
>>> applicant = create_applicant(user)
DEBUG Making POST request to https://api.onfido.com/v3/applicants
DEBUG <Response [201]>
DEBUG {u'first_name': u'hugo', u'last_name': u'rb', u'middle_name': None, ...}
DEBUG Creating new Onfido applicant from JSON: {u'first_name': u'hugo', u'last_name': u'rb', ...}
<Applicant id=a2c98eae-XXX user='hugo'>

2. Create your check + reports for the applicant:

>>> from onfido.helpers import create_check
>>> create_check(applicant, 'standard', ['identity', 'right_to_work'])
>>> assert Check.objects.count() == 1
>>> assert Report.objects.count() == 2

This will create the Check and Report objects on Onfido, and store them locally as Django model objects.

3. Wait for callback events to update the status of reports and checks:

DEBUG Received Onfido callback: {"payload":{...}}
DEBUG Processing 'check.completed' action on check.bd8232c4-...

NB If you are using the callback functionality, you must set the ONFIDO_WEBHOOK_TOKEN property (see settings section below). The callback handler will force verification of the X-Signature request header as specified in the webhooks documentation.

The raw JSON returned from the API for a given entity (Applicant, Check, Report) is stored on the model as the raw attribute, and this can be parsed into the relevant model attributes. (Yes this does mean duplication of data.) The core pattern for interaction with the API on a per- object basis is a read-only fetch / pull pattern (analagous to git operations of the same name). If you call the fetch method on an object, it will use the href value in the raw JSON to fetch the latest data from the API and parse it, but without saving the changes. If you want to update the object, use the pull method instead.

The Report object is a special case, where the raw data from the API often contains sensitive information that you may not wish to store locally (passport numbers, Visa information, personal data). In order to get around this, there is a scrub_report_data function that will remove certain attributes of the raw data before it is parsed. By default this will remove the breakdown and properties elements.

>>> check = Check.objects.last()
>>> check.raw
{
    "id": "c26f22d5-4903-401f-8a48-7b0211d03c1f",
    "created_at": "2016-10-15T19:05:50Z",
    "status": "awaiting_applicant",
    "type": "standard",
    "result": "clear",
    "href": "applicants/123/checks/456"
}
>>> check.fetch()  # fetch and parse the latest raw data
>>> check.pull()  # calls fetch and then saves the object

There is a management command onfido_sync which can be used to pull all the objects in a queryset. It takes a single positional arg - 'applicant', check' or 'report', and has two options - --filter and --exclude - both of which take multiple space-separated args which can be used to manage the queryset that is used.

Examples:

$ ./manage.py onfido_sync check
$ ./manage.py onfido_sync report
$ ./manage.py onfido_sync check --filter complete
$ ./manage.py onfido_sync check --exclude complete

Settings

The following settings can be specified as environment settings or within the Django settings.

• ONFIDO_API_KEY: your API key, found under setting in your Onfido account.
• ONFIDO_WEBHOOK_TOKEN: (optional) the Onfido webhook callback token - required if using webhooks.

The following settings can be specified in the Django settings:

• ONFIDO_LOG_EVENTS: (optional) if True then callback events from the API will also be recorded as Event objects. Defaults to False.
• ONFIDO_REPORT_SCRUBBER: (optional) a function that is used to scrub sensitive data from Report objects. The default implementation will remove breakdown and properties.

Tests

If you want to run the tests manually, install poetry.

$ poetry install
$ poetry run pytest

If you are hacking on the project, please keep coverage up.

Contributing

Standard GH rules apply: clone the repo to your own account, make sure you update the tests, and submit a pull request.

django-onfido's Issues

Improve admin views

Better sorting / filtering / searching.

[ ] Search applicants, checks, reports by onfido_id
[ ] Search applicants, checks, reports by user first / last name
[ ] Filter checks, reports by result
[ ] Display applicant user.get_full_name() instead of username

Migrate from travis-ci to GH actions

Support Django 3.1

Django admin 500 when searching through Onfido.Event

The resource_id field was renamed and migrated to onfido_id but the Admin.search_fields attribute was never updated.

It simply needs to be renamed to onfido_id.

I would submit a PR but I'm aware of the current 1.9 blocker, let me know how fixing this will fit in to that.

Pull latest data on event update

The lib currently doesn't update objects when an incoming event is received. This was mainly to prevent the situation of an update not being processed because the API is down, but this is probably overly-cautious (after all, they just sent an event) - however, it should be handled gracefully.

Now that the inbound webhook has been tested in production, we should update the update_status method to pull the latest.

Scrub sensitive data

When calling the pull() method on a report it will pull down a bunch of raw data that you may not want to store locally in the raw field. (e.g. the breakdown and properties fields). We should include a setting to be able to scrub the data before saving.

Preferred option at the moment is to use a function that can be used 'scrub' the data.

For example - this is a typical UK right to work report JSON value:

{
  "breakdown": {
    "data_validation": { ... },
    "image_integrity": { ... },
    "police_record": { ... },
    "right_to_work": { ... },
    "visual_authenticity": { ... }
  },
  "created_at": "2016-10-24T08:40:03Z",
  "href": "/v2/checks/123/reports/456",
  "id": "456",
  "name": "right_to_work",
  "properties": {
    "date_of_birth": "1990-01-01",
    "date_of_expiry": "2020-01-01",
    "document_numbers": [
      {
        "type": "document_number",
        "value": "1234567890"
      }
    ],
    "document_type": "passport",
    "first_name": "Fred",
    "gender": "Male",
    "issuing_country": "GBR",
    "last_name": "Flintstone",
    "mrz_line1": "F        ",
    "mrz_line2": "123456789<<<<<<<<<<<<<<XX",
    "nationality": "GBR"
  },
  "result": "clear",
  "status": "complete",
  "variant": "standard"
}

The default scrubber will remove the entire breakdown and properties blocks:

def scrub_report(data):
    del data['breakdown']
    del data['properties']

Running this against the full JSON would give:

{
  "created_at": "2016-10-24T08:40:03Z",
  "href": "/v2/checks/123/reports/456",
  "id": "456",
  "name": "right_to_work",
  "result": "clear",
  "status": "complete",
  "variant": "standard"
}

In order to customise the function, write your own, and then set the ONFIDO_REPORT_SCRUBBER setting:

# settings.py

def custom_scrubber(data):
    """Remove breakdown and document numbers only."""
    del data['breakdown']
    del data['properties']['document_numbers']

ONFIDO_REPORT_SCRUBBER = custom_scrubber

pip refuse to install 1.0+ with python 3.7 without option --ignore-requires-python

pip install git+git://github.com/yunojuno/[email protected]

ERROR: Package 'django-onfido' requires a different Python: 3.7.3 not in '<4.0,>=3.8'

Probably a packaging issue since v1.0 tests are OK with python3.7.3.

collected 54 items                                                                                                                         

tests/test_admin.py .                                                                                                                [  1%]
tests/test_api.py ....                                                                                                               [  9%]
tests/test_decorators.py ...                                                                                                         [ 14%]
tests/test_helpers.py ...                                                                                                            [ 20%]
tests/test_integration.py .                                                                                                          [ 22%]
tests/test_settings.py ....                                                                                                          [ 29%]
tests/test_views.py .                                                                                                                [ 31%]
tests/test_models/test_applicant.py ....                                                                                             [ 38%]
tests/test_models/test_base.py ..................                                                                                    [ 72%]
tests/test_models/test_check.py ...                                                                                                  [ 77%]
tests/test_models/test_event.py .....                                                                                                [ 87%]
tests/test_models/test_report.py ...                                                                                                 [ 92%]
tests/test_admin.py ...                                                                                                              [ 98%]
tests/test_data.py .                                                                                                                 [100%]

Update to support v3 API

There is a new v3 API.

Remove stored `is_clear` field

Replace with a property which returns True/False dependent on the result status (being CLEAR)

Add support for webhook X-Signature verification

From the docs:

Each webhook URL is associated with a secret token. This is visible in the response.token field when registering a webhook.

Events sent to your application will be signed using this token: verifying the request signature on your server prevents attackers from imitating valid webhooks. The HMAC digest signature, generated using SHA-1, will be stored in a X-Signature header.

When you receive an event, you should compute a hash using your secret token, and ensure that the X-Signature sent by Onfido matches that hash.

The webhook handling view needs to be able to verify these tokens.

Support Django 4

Django 4 was released today, but django-onfido advertises itself as compatible with Django >= 3.1 and < 4.0.

Update webhook to use SHA256

Update the verification process:

https://developers.onfido.com/guide/manual-webhook-signature-verification

Add support for native JSONField and Django 1.9, 1.10

Override check result in case of 'consider'

When a check is complete, it can have one of two results - 'clear', which means all reports came back clear, and 'consider', which means that one or more of them did not. In this it is up to the users of the lib to decide whether to pass the check or not. e.g. if an identity check fails because an uploaded passport photo is bad, that doesn't mean they fail the test, just that it can't pass automatically. However, if you are restricting someone's use of your app because of their check status, this poses a problem:

if check.result != 'clear':
    print "You cannot proceed"

Obviously you could just overwrite the result with 'clear' but you are then amending the history, and this is not ideal. In the event of an audit you need to be able to demonstrate what happened. Alternative is an additional field that can be used to declare the check as clear, outside of the callbacks. I think this is my preferred option - a NullBooleanField that is None by default, set to True automatically if a check is updated via the API to 'clear', but then overridable in the admin.