A Rust implementation of Kiln (the execution framework, formerly the Spark Execution Framework) — the execution pillar for running autonomous AI development work, with the NVIDIA DGX Spark as its reference substrate.
kiln-cli is the developer switch and executor loop. It publishes a
CapabilityManifest (what can run here), admits frozen WorkUnits, runs their
sealed cell-DAGs through a verification gate, and emits VerdictEvents —
computed-done, never claimed-done. The whole thing is
spec-driven: the domain model (the What), the architecture (the How), and
the behavioural conformance suite all live under .product/ and are kept in lockstep
with the code.
The Spark is bandwidth-bound and its VRAM holds one residency at a time. The developer throws a switch between two mutually exclusive modes:
| Mode | What runs | Shape |
|---|---|---|
| QUEUE | many small models, batched inference | high-throughput, many units in flight |
| EXPLORER | one large model, serial | deep, single discovery session |
| OFF | nothing resident | the default |
The flip is a deliberate human act — no machine-rate process triggers it, because swapping a residency is expensive. (Opus-class OFF-BOX work is not a box mode; it lives elsewhere.)
- WorkUnit — travels in by-value, identified by
bundle_hash. Its cells form a sealed DAG. Binding-homogeneity invariant: every cell of a unit requires the same SPMC model binding (model identity + served quantization + params). Quantization is load-bearing on the Spark; a heterogeneous unit is a decomposition defect and is never dispatched. - CapabilityManifest — the executor publishes its self-description out of
band (
kiln manifest): thebindingsit can serve, thedeliverymodes/schemes/ integration-methods/forges, theshape-languagesandgate-kindsit runs. A producer matches a unit's derived requirements against it before dispatch — empty distance ⇒ executable. It is the one seam artifact the executor authors. - Admission — a distinct gate before verification. A unit whose derived
requirements the box cannot cover never runs — it is answered
not-admittedwith the concretemissing-capabilitiesdistance, binding tohalt(a higher tier never adds a missing capability). - Verification — each cell is gated by a protected oracle the worker cannot
write (ADR-076). A worker that can write its own oracle has no verifier. Verdicts
are
accepted/rejected/escalate/not-admitted; consequences areadvance/halt/retry/escalate. Escalation is unit-atomic — the whole unit moves one binding up the ladder, never a single cell. - Artifact delivery — a unit declares where produced work lands:
inline(artifact bodies by value in the verdict) orrepository(a declared git repo —file:///local, remote for production — landed viapush-branchorpull-request; the verdict carriesdelivery-result: branch, commit,pr-url). No credential material ever rides in the WorkUnit — repository/forge access is exchanged executor-side from thecredential-grantreference. - VerdictEvent — travels out fire-and-forget to a durable, append-only log.
interface WorkUnit / Cell / ModelBinding / VerdictEvent / CapabilityManifest (by-value contract, bundle_hash)
switch Box Control — the developer switch (QUEUE ⇄ EXPLORER), distinct-mode guard
queue Work-Unit Queue — admission (homogeneity guard), priority, escalation ladder
execution Execution — sealed cell-DAG walk, verdict reduction, + oracle-run gate
exploration Exploration — single serial discovery session (EXPLORER)
serving Model Serving — VRAM residency + batched inference + Worker seam ← production
sandbox Isolation — per-unit ephemeral sandbox + brokered credentials ← production
stream Verdict Stream — durable, append-only, idempotent verdict log ← production
host Serving Host — vLLM residency materialized on the box over SSH ← production
executor Engine — composes everything; persists to .kiln/state.json
cli kiln / kiln-conform binaries
Each bounded context is one crate. Every aggregate is a decider
(decide() -> Result<Vec<Event>, &'static str>, where Err is the violated
invariant id); every read-model is a projector (an event fold).
cargo build --release # produces target/release/kiln and kiln-conform
cargo test # 97 tests across the workspacekiln manifest # publish the CapabilityManifest (what can run here)
kiln mode set queue # throw the developer switch into QUEUE
kiln admit work-unit.json # admit a frozen WorkUnit (structural + capability pre-flight)
kiln run # drain the queue (in-memory demo path)
kiln serve # drain isolated: sandbox + creds + worker + oracle + durable log
kiln status # box mode + read-model views
kiln stream # print the emitted VerdictEvents
kiln mode set explorer && kiln explore # run a discovery session (EXPLORER only)A WorkUnit is the canonical contract JSON — kebab-case throughout (unit-ref,
spmc-bundle, cell-graph, acceptance-class ∈ auto-commit-if-green |
needs-verdict, artifact-delivery ∈ inline | repository). See
examples/workunit-csharp.json (inline) and
examples/workunit-csharp-repo.json
(repository), plus docs/production-seams.md for the
full kiln serve pipeline.
State persists to .kiln/state.json; the durable verdict log to
.kiln/verdicts.jsonl; per-unit sandboxes under .kiln/sandboxes/.
kiln serve runs each unit's frontier through a Worker and gates it with a
protected Oracle — both wired by environment variable:
# 1. an OpenAI-compatible model server on the box (llama-server / vLLM / TGI)
export KILN_OPENAI_BASE_URL=http://127.0.0.1:8080 # → built-in OpenAiWorker
export KILN_OPENAI_MODEL=qwen2.5-coder-7b # optional
# 2. the protected gate the worker cannot write (ADR-076)
export KILN_ORACLE_CMD='cargo test --quiet'
kiln mode set queue && kiln admit unit.json && kiln serveWorker precedence: a residency materialized by kiln mode set → KILN_OPENAI_BASE_URL
(HTTP) → KILN_WORKER_CMD (shell) → offline StubWorker. Full box setup:
docs/running-on-kiln.md.
When KILN_SSH_TARGET is set, kiln mode set physically materializes the
residency: it retires any live host, then launches the mode's model as a vLLM
container on the box over SSH, polls its /v1 endpoint, and only serves once it
answers. The switch becomes a real start/stop of VRAM, not a flag — and kiln serve then auto-targets that host.
export [email protected] # → built-in SshVllmHost backend
export KILN_QUEUE_MODEL=qwen2.5-coder-7b # model vLLM loads in QUEUE
kiln mode set queue # launches the container, waits until readyThe implementation is derived from, and continuously checked against, a Product Framework model:
- What — 8 bounded contexts, 13 entities (11 aggregate roots, each a decider), events / commands / invariants / read-models.
- How — 9 decisions, 10 principles, 12 patterns; the application contract and the Rust crate layout.
- Deciders & projectors — every aggregate's guarded state machine, proven sound & complete by simulation against its scenarios.
- Behavioural conformance (§6.3) —
kiln-conformreplays the realised Rust deciders against the spec's scenario oracle. All 11 deciders are conformant. - Deliverables — acceptance criteria wired to named, passing
cargo tests and computed-done.
Re-run the gates:
product domain validate # What graph conformant
product how validate # How contract conformant
product archetype check kiln-cli # crate layout matches the tree
CONF="$PWD/target/release/kiln-conform"
product decider conform box-decider --runner "$CONF box-decider" # §6.3, per decider
product deliverable done deliverable-serving # computed-done %- 79/79 tests pass · release build green
- 11/11 deciders behaviourally conformant
- 11/11 deliverables computed-done
- domain / how / archetype all conformant
Physical infrastructure (GPU model serving, microVM isolation, the on-box vLLM
container) is implemented behind the Worker, SandboxRuntime, CredentialBroker,
and ResidencyHost trait seams with working local backends — a real server, a
container runtime, or the SSH/vLLM host drops in without touching the spec.