tishka17 / clepsydra Goto Github PK

We need to pass some context to any task.
It should contain

• scheduler instance
• info about task run (run type, retry and so on)
• custom global params
• params from middlewares

Task can be planned for one time run or for periodic run.
For periodic tasks we expect:

• delay (or time) before first run
• period between repeats:
  • monthly on certan date
  • weekly
  • after certain number of days
  • every N seconds or minutes
  • cron-like syntax

Tasks must be started concurrently with limitation of number

You need to make a timer test feature

It' s especially useful for cron-timers, because their logic is the most complex.

Example API:

from datetime import datetime

import pytest
from clepsydra.testing import create_test_scheduler, TestScheduler
from clepsydra.rules import CronRule


def first_job(context: dict):
    context["tested"] = True


@pytest.fixture
def scheduler() -> TestScheduler:
    scheduler = create_test_scheduler()
    scheduler.task(first_job)

    scheduler.add_task(
        "first_job",
        CronRule(hour="*", minute=0, second=0)
    )


def test_first_job(scheduler: TestScheduler) -> None:
    context = scheduler.trigger_tasks(datetime(2021, 3, 3, 5, 0, 0))
    assert context.get("tested")

This API is really far from perfect, so I welcome the discussion.

Tasks must be started concurrently with different limitation strategies: by number, by memory and so on

Configure interval per-scheduer and per-job when missed jobs can be executed.
Check different reasons of missing: because of other jobs or because of stopped scheduler

Support to run multuple workers on different severs with single task storage