mozilla / django-post-request-task

A celery task class whose execution is delayed until after the request finishes, using request_started and request_finished signals from django.

django-post-request-task

A celery task class whose execution is delayed until after the request finishes, using request_started and request_finished signals from django and thread locals.

This is useful if your views are wrapped in transactions (as they should if you're making database modifications in them), as you can end up triggering a celery task too soon before the transaction has been committed (or even trigger a task when the corresponding transaction has been rolled back).

By listening to the request_started and request_finished django signals, we can safely trigger a task after all transactions created from @atomic or ATOMIC_REQUESTS have been committed.

Usage

from celery import Celery
from post_request_task.task import PostRequestTask


app = Celery('myapp', task_cls=PostRequestTask)

@app.task
def my_task():
    # If .delay() is called on this task inside a django request-response
    # cycle it will be called once the request is finished, and not before.
    pass

Or, if you are using the decorator directly:

from post_request_task.task import shared_task

@shared_task
def my_task():
    pass

That's it. If the task is called from outside the django request-response cycle, then it will be triggered normally.

As a bonus feature, if the same task is called with the same argument several times during a request-response cycle, it will only be queued up once.

Running tests

$ make testenv
$ make test

By default, tests are run with whatever django version is installed. If you want to run tests for other versions use tox:

$ make testenv
$ tox -e 3.7-2.0.x # or any other environment defined in our tox.ini

django-post-request-task's Issues

Return an explicit and useful object in `apply_async()`

The return value of PostRequestTask.apply_async(), when task queuing is enabled, is None instead of an AsyncResult. It makes sense since the task has not been sent yet, but it's annoying when you want to do something with the AsyncResult...

So ideally, we should return some kind of object instead, like a PostRequestTaskAsyncResult, which would be a proxy for AsyncResult. In that proxy object, accessing any property should cause us to bypass the task queuing, triggering the original apply_async() method, sending the task to the broker instead of waiting for the request to finish. We could then grab the AsyncResult and return the value of the property being accessed.

Error when `pip install`

ERROR: django-post-request-task 0.4.0 has requirement Django<=2.2,>=1.11, but you'll have django 2.2.3 which is incompatible.

In setup.py we have:

install_requires = [
    'Django>=1.11,<=2.2',
    'celery>=3.0,<5.0',
]

Can you please change it to:

    'Django>=1.11,<2.3',
    'celery>=3.0,<5.0',

and release a new version?

Please distribution for Python3 on PyPI

On PyPI the only available file for distribution is: django_post_request_task-0.0.3-py2-none-any.whl. (Notice py2 in URL.)

The project's tests pass in a Python3 venv, so it looks like it is Python3 compatible. Can you provide a package that is Python3 compatible so the package may be used in Python3 projects?

(If there is remaining work to make the package Python3 compatible, I might be able to help.)

Below demonstrates results under different versions of Python.

Thanks!

Python2

$ virtualenv testv
New python executable in testv/bin/python2
Also creating executable in testv/bin/python
Installing setuptools, pip...done.

$ ./testv/bin/pip install django-post-request-task
You are using pip version 6.0.8, however version 8.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting django-post-request-task
  Downloading django_post_request_task-0.0.3-py2-none-any.whl
Installing collected packages: django-post-request-task

Successfully installed django-post-request-task-0.0.3

Python3

$ pyvenv testv

$ ./testv/bin/pip install django-post-request-task
You are using pip version 7.1.0, however version 8.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting django-post-request-task
  Could not find a version that satisfies the requirement django-post-request-task (from versions: )
No matching distribution found for django-post-request-task

Add circle badge

It would be nice to have a circle ci badge on the README.

Django 3.0 support

question:

This package looks awesome, the activity looks low since django3.0 came out, Would you say it is stable with django 3.0?

Travis CI free usage ends Dec 3; mozilla repos should switch to other CI platforms

We're opening this issue because your project has used Travis CI within the last 6 months. If you have already migrated off it, you can close and ignore this issue.

Travis CI is ending free builds on public repositories. travis-ci.com stopped providingthem in early November, and travis-ci.org will stop after December 31, 2020. To avoid disruptions to your workflows, you must migrate to another CI service.

For production use cases, we recommend switching to CircleCI. This service is already widely used within Mozilla. There is a guide to migrating from Travis CI to CircleCI available here.

For non production use cases, we recommend either CircleCI or Github Actions. There is a guide to migrating from Travis CI to Github Actions available here. Github Actions usage within Mozilla is new, and you will have to work with our github administrators to enable specific actions following this process.

If you have any questions, reach out in #github-admin:mozilla.org on matrix.

CODE_OF_CONDUCT.md file missing

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please see Mozilla-GitHub-Standards or email [email protected].

(Message COC001)

Move travis to xenial, drop Django 1.8 and Python 3.5

Replace legacy convenience image in CircleCI config

circleci/* legacy images are deprecated and no longer maintained, they need to be replaced by the more modern cimg/* images. See https://discuss.circleci.com/t/legacy-convenience-image-deprecation/41034 for more information.

django-post-request-task/.circleci/config.yml

Line 6 in 574ede5

- image: circleci/python:<< parameters.python >>

Celery 5 support

Thanks for elegant solution of celery/django/transactions integration.

Celery released new major version 5.0.0 on September, 24 2020. It would be nice to see it's support in this project, but celery version limited in setup.py

Wiki changes

FYI: The following changes were made to this repository's wiki:

• defacing spam has been removed

• Restricting write access to contributors is strongly encouraged. Please make that change (documentation).

These were made as the result of a recent automated defacement of publically writeable wikis.