dsoftwareinc / django-rq-scheduler

Package migrated to https://github.com/dsoftwareinc/django-tasks-scheduler

Django RQ Scheduler

This project has been migrated to django-tasks-scheduler.

django-rq-scheduler's Issues

Django 4.0 warnings

Describe the bug
In Django 4.0 apps do not need to declare a default_app_config in the init.py file.

To Reproduce
Steps to reproduce the behavior:

1. Add scheduler to your INSTALLED_APPS

Expected behavior
No warnings when the project runs

Improve JobArg/JobKwarg

Is your feature request related to a problem? Please describe.
Currently have 3 possible fields. Should have one field and be implemented.

Add ability to test method before saving job

Is your feature request related to a problem? Please describe.
Add the ability to test method before saving a job in django-admin

Add option to not depend on rq-scheduler

Is your feature request related to a problem? Please describe.

RQ (already from version 1.2.0, now on 1.11.1) provides support for a worker (or many workers) to also behave as schedulers (1 active at a time). The main motivation is cost, having a scheduler for each project means an extra container/pod

Django-RQ already supports this, with get_scheduler() being either a Scheduler (rq-scheduler) or fallback as a Queue

Describe the solution you'd like
I'd like to contribute on how to offer that kind of fallback to django-rq-scheduler.

Describe alternatives you've considered
I believe the ability to display and manage jobs is (or should allow to be) separate from the task of running the scheduler itself.

Run Now not running when job is already scheduled

Describe the bug
"Run Now" has no effect for jobs that are already scheduled (or cron'ed with the new cron)

One option would be to replace in admin.py, line 113: enqueue_at( ...
with enqueue(... (remove the first parameter)

Thanks!!

Run scheduler in management command

Is your feature request related to a problem? Please describe.
Instead of having a threaded scheduler - have it run on management command.

Stale scheduled jobs (whose responsibility, what to do about it?)

Hi!

I'm testing manually the resiliency of the new cron (2023.1+).

I came across a "(cron)scheduled job" that is scheduled in the past. Other jobs are picked up from that registry and queued, but this job doesn't.

Question 1: does the job exist? (or is it an artifact)
Question 2: is it RQ's responsibility? (and for some reason it's not picking it up?)
Question 3: can we do something about it? (e.g enqueue it if we see that it's X time old, and the queue is empty and there's 0 active jobs in that queue? (all things we can know about with certainty))

Thanks!

Scheduler jobs event log in django-model

Is your feature request related to a problem? Please describe.
Create a log of events in django-rq-scheduler.

• Store when a job got scheduled and for when it was scheduled.
• Store the job run instances and results in the database.
• Purge log based on configuration.

Add import/export jobs button to admin

• Add export jobs button
• Add import jobs button

Scheduling RepeatedJob with first time in the past

Describe the bug
See discussion here

User wants a Repeated job to be executed forever starting from 2022/01/01 every 1h. What time should be placed as scheduled_time in this case? If you put there, for example, a very distant day in the future (e.g. 2050/01/01) the django-rq-scheduler will create a job, but RQ scheduler will be waiting till the 2050/01/01 to get it executed for the first time.

To Reproduce
Schedule a RepeatedJob to run every 1h starting from a time in the past.

Expected behavior
Should be scheduled if there is an expected run in the future.

The max_length of queue name is too short

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Go to settings and add the queue with a name more than 16 characters long.
2. Try to run the project
3. See error:

ERRORS:
scheduler.CronJob.queue: (fields.E009) 'max_length' is too small to fit the longest value in 'choices' (33 characters).
scheduler.RepeatableJob.queue: (fields.E009) 'max_length' is too small to fit the longest value in 'choices' (33 characters).
scheduler.ScheduledJob.queue: (fields.E009) 'max_length' is too small to fit the longest value in 'choices' (33 characters).

Expected behavior
Need to extend the queue name size up to 255 characters (default value) and project have to run successfully.

Desktop (please complete the following information):

• OS: Linux Ubuntu
• python version 3.10.11
• django version 4.2.1
• django-rq-scheduler version 2023.5.0

Additional context
Actually I just need to add the previously created long queue names which cannot be changed.
The problem is in BaseJob class here

Thank you,
Ihor

Cannot install Django-rq-scheduler with pip-tools

I manage my requirements with pip-tools. I do a pip-compile on my requirements.in to create a requirements.txt, then a pip-sync to update the env. Works fine, usually.

Now I added django-rq-scheduler to the reqs and I get a weird error:

❯ pip-compile
WARNING: the legacy dependency resolver is deprecated and will be removed in future versions of pip-tools. The default resolver will be changed to 'backtracking' in pip-tools 7.0.0. Specify --resolver=backtracking to silence this warning.
Traceback (most recent call last):
  File "/Users/mjm/Dropbox/Devel/ACHN/Vraagbaak/venv/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/Users/mjm/Dropbox/Devel/ACHN/Vraagbaak/venv/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mjm/Dropbox/Devel/ACHN/Vraagbaak/venv/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/mjm/Dropbox/Devel/ACHN/Vraagbaak/venv/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mjm/Dropbox/Devel/ACHN/Vraagbaak/venv/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mjm/Dropbox/Devel/ACHN/Vraagbaak/venv/lib/python3.11/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mjm/Dropbox/Devel/ACHN/Vraagbaak/venv/lib/python3.11/site-packages/piptools/scripts/compile.py", line 580, in cli
    results = resolver.resolve(max_rounds=max_rounds)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mjm/Dropbox/Devel/ACHN/Vraagbaak/venv/lib/python3.11/site-packages/piptools/resolver.py", line 253, in resolve
    has_changed, best_matches = self._resolve_one_round()
                                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mjm/Dropbox/Devel/ACHN/Vraagbaak/venv/lib/python3.11/site-packages/piptools/resolver.py", line 343, in _resolve_one_round
    their_constraints.extend(self._iter_dependencies(best_match))
  File "/Users/mjm/Dropbox/Devel/ACHN/Vraagbaak/venv/lib/python3.11/site-packages/piptools/resolver.py", line 456, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mjm/Dropbox/Devel/ACHN/Vraagbaak/venv/lib/python3.11/site-packages/piptools/repositories/local.py", line 87, in get_dependencies
    return self.repository.get_dependencies(ireq)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mjm/Dropbox/Devel/ACHN/Vraagbaak/venv/lib/python3.11/site-packages/piptools/repositories/pypi.py", line 237, in get_dependencies
    wheel_cache = WheelCache(self._cache_dir, self.options.format_control)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: WheelCache.__init__() takes 2 positional arguments but 3 were given

I tried fixing the version, but no effect.

Doing just a plain pip install Django-rq-scheduler works normally. So it might also be a pip-tools problem, although pip-tools is mostly well-behaved.

Package incompatible with >=Django 3.2

Describe the bug
According to the changes in PIP (see changes-to-the-pip-dependency-resolver) now django-rq-scheduler shows as incompatible during install with >=Django 3.2

To Reproduce

1. Create requirement file with, for example, django==3.2.1 django-rq-scheduler==2022.5
2. pip install -r requirements.txt
3. Error message:

The conflict is caused by:
The user requested django==3.2.1
django-rq-scheduler 2022.5 depends on django>=4.0

Expected behavior
Installed successfully as package supports Django>=3.2

Desktop:

• OS: any
• pip: >=21.0

at_front via kwargs not working

(cross-posted also to django-rq):

I have this setting in the job, and still cannot get this job to be processed first, namely to be placed at the front of the queue.

I'm using django-rq, with django-rq-scheduler. In "Cron jobs" page I set the job's details as this:

And as per the first image, it seems to have been picked up by django-rq.

Any ideas/help would be appreciated.

2023.1+ "cron" explanation + re-scheduling of cron jobs not working

HI @cunla !
I'm successfully deploying 2023.1 , and as far as I've seen, it replaces the rq-scheduler's "cron()" with a smart way of scheduling the next run.

I wanted to make sure about a few things:

1. what happens if I manually remove the crons that are "scheduled"? is there a recovery mechanism? (this is not an issue)

2. is there a job de-duplication check that, for example, if the job is in the "scheduled" registry, going to the Cron section and clicking on "run now" will have no effect, apparently.

3. what If I go to the scheduled registry for a job and click "Enqueue". then it will run, but stay in the scheduled registry (at least for a while) as "finished". what about the next run, do I have to disable the cron and re-enable to have the "next" run?

Perhaps I could help with some documentation/readme after I have mastered how it works?

Work-horse terminated unexpectedly; waitpid returned 139 (signal 11); <-- scheduler.tools.run_job

Describe the bug
Once or twice per day, a job that runs every minute fails like this:
Work-horse terminated unexpectedly; waitpid returned 139 (signal 11);
(Upon further investigation, signal 11 is a SIGSEGV Error (Address boundary error)).

scheduler.tools.run_job does for me: only enqueues the real job.

Desktop (please complete the following information):

• OS: alpine-3.17
• python 3.11
• django 4.1.7
• django-rq 2.7.0
• django-rq-scheduler 2023.3.2

Additional context
I just want to see if this happens to anyone, and what are the possible steps to isolate it and fix it.

Thanks!

make at_front nullable in the Model

Is your feature request related to a problem? Please describe.
When I tried to import cron jobs (in a json file, from a pre-at_front version) it failed.
It would be an easy backward-compatible fix to allow at_front to be nullable. (and to cast to bool as needed)

Remove need for rq-scheduler

Is your feature request related to a problem? Please describe.
Remove need for rq-scheduler.
django-rq has internal scheduler since version 1.2.0.

Dumping the jobs to a json does not dump the arguments

Describe the bug / enhancement
Using Django-smuggler, etc, when downloading the data it does not export the arguments. It would be very helpful to have at least some metadata exported that if imported would regenerate the job the way it's supposed to be.

To Reproduce
Steps to reproduce the behavior:

1. Go to Dump data (django-smuggler)

Expected behavior
a downloaded json that upon importing it back it would also import the parameters

js/base.js is referenced but not in the repo

Describe the bug

The scheduler ModelAdmin tries to load /static/scheduler/js/base.js (defined here) but this file is missing from the repo.

Question

Hey, I ran into this nice project when researching alternative of run django scripts in cron, a few questions:

• are you rebranding this to https://github.com/dsoftwareinc/django-tasks-scheduler ? (which makes sense but just to confirm here)
• is the plan to continue use rq and be compatible with django-rq ?
• also currently this seems to hit db every time reschedule in worker, which is great for consistency. However, given the fact that, for most apps, db are usually slower and busier than redis, would you consider further improvement or alternative to reduce the use of db (maybe only when worker initializing and user add/update/remove jobs)?

'ScheduledJob' object has no attribute 'all'

Describe the bug
Can't delete one job with admin.delete_model

To Reproduce
Steps to reproduce the behavior:

1. Go to scheduledjob in Django admin
2. Click on any job
3. Scroll down to bottom
4. Click "Delete"
5. Click "Yes, I'm sure"
6. See error

Expected behavior
Delete scheduled job

Screenshots

Desktop (please complete the following information):

• OS: MacOS
• python version: 3.9.13
• django version: 4.0.5
• django-rq version: 2.5.1
• requirements.txt?: -

Additional context

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 683, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 133, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/cache.py", line 62, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/sites.py", line 242, in inner
    return view(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 133, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 2104, in delete_view
    return self._delete_view(request, object_id, extra_context)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 2141, in _delete_view
    self.delete_model(request, obj)
  File "/usr/local/lib/python3.9/site-packages/scheduler/admin.py", line 68, in delete_model
    for obj in queryset.all().iterator():

Exception Type: AttributeError at /admin/scheduler/scheduledjob/2/delete/
Exception Value: 'ScheduledJob' object has no attribute 'all'

Calculate kwargs at runtime

Is your feature request related to a problem? Please describe.
Add ability to calculate kwargs at runtime

Add to admin visibility whether there is a scheduler running

Is your feature request related to a problem? Please describe.
Currently there is no way to know whether there is a worker with a scheduler.

Describe the solution you'd like
Add a check whether there is a worker with a scheduler and make it visible in the admin view.

Refactor tests

• Separate tests from test-tools.
• Test multiple schedulers.