olydis / autorest-ci Goto Github PK

Goal:

• no race conditions (multiple workers stealing same job, swallowed PR update), but also no heuristical race condition checks
• immediate CI icon appearance
• requesting CI restart

Solution: two jobs

1. Watcher/Scheduler

• run exactly on instance, centrally
• keeps STATE about repos.
  • TODO: draw state machine
  • hammers through all repos and PRs and classifies, e.g. "pending":"[assign {SESSION UID}] New PR", "pending":"[assign {SESSION UID}] Updated PR", "pending":"[assign {SESSION UID}] Previous worker looks dead"
  • if in state "[assign]" and sees "[assign_me {SESSION UID} {WORKER ID}]" => set "[launch {SESSION UID} {WORKER ID}]"
  • if in state "[launch]" and see message not by "{SESSION ID}", hammer over it with "[launch]". Leave this state (into "observant {WID} {SID}" state) only if message by expected session seen.
  • waits for status by worker indicating success/failures => transition into such state, strip away "[...]" and actually send that status (and continue to do so if anyone overwrites)

2. Worker

• only ever sends "pending" status!
• iterates PRs and checks for:
  • [assign {SESSION UID}] => [assign_me {SESSION UID} {WORKER ID}]
  • [launch {MY WORKER ID} {SESSION ID}] => while running [running {MY WORKER ID} {SESSION ID}] status

TODO: create Coq proofs for properties:

• new/updated/stalled PRs lead to assignment phase
• for each PR, if pending => if worker exists, worker will be assigned
• at most 1 worker working on PR