Kev POC

Self-host Kev-4B, the open-weight SystemOne decision model from jared-palmer/kev, behind a TypeSafe-compatible POST /v1/systemone API, with one command:

docker compose up --build -d

The container runs the maker's own kev.serve server (CUDA torch + flash-linear-attention when available), downloads the adapter + Qwen3.5-4B-Base weights on first start, and caches them in the kev-hf volume.

Check it's up

curl -s localhost:8008/v1/models

Connect from other machines

The container is exposed on all host interfaces (0.0.0.0:8008 in docker-compose.yml), so any machine that can reach this host can call it:

curl -s http://<this-host-ip>:8008/v1/models
curl -s http://<this-host-ip>:8008/v1/systemone -H 'content-type: application/json' -d '...'

Find this host's IP with hostname -I. In the TypeSafe SDK, point base_url at the host instead of 127.0.0.1. On WSL2 the IP can change on reboot (the Windows host auto-forwards localhost to WSL), and Windows Firewall may need to allow inbound TCP 8008.

Security: with no KEV_API_KEY set the API is open to anyone who can reach port 8008. Set KEV_API_KEY (see Configuration) before exposing it beyond a trusted network.

Call it (curl)

curl -s localhost:8008/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
  "model": "kev-latest",
  "questions": {
    "department":  {"type": "choice", "instructions": "Which team should handle this?",
                    "criteria": {"returns": "Exchanges, refunds, wrong or damaged items",
                                 "shipping": "Delivery status, delays, lost packages",
                                 "billing": "Charges, invoices, payment problems"}},
    "escalate":    {"type": "noul",  "instructions": "Does this need urgent human attention?"},
    "frustration": {"type": "score", "instructions": "How frustrated is the customer?",
                    "criteria": ["Calm", "Frustrated", "Very angry"]}
  }}'

Call it from the TypeSafe SDK

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

client = TypeSafeClient(
    api_key="local",
    base_url="http://127.0.0.1:8008",
    model="kev-latest",
)
result = client.system_one(
    "I was charged twice. Please fix this ASAP.",
    {
        "billing": Noul(instructions="Is this ticket about billing?"),
        "department": Choice(
            instructions="Which team should handle this?",
            criteria={"billing": "Payments", "shipping": "Delivery", "access": "Login"},
        ),
        "urgency": Score(
            instructions="How urgent is this ticket?",
            criteria=["can wait", "this week", "today"],
        ),
    },
)
print(result.nouls["billing"].noul)
print(result.choices["department"].choice)
print(result.scores["urgency"].score)

Configuration

Variable	Default	Effect
`KEV_MODEL`	`jaredpalmer/kev-4b`	Hub repo/checkpoint to serve (`kev-0.8b`, `kev-9b`, `kev-27b`...).
`KEV_BACKEND`	`cuda`	Build-time arg: `cuda` or `cpu` (CPU boxes build with `KEV_BACKEND=cpu`).
`KEV_DTYPE`	`bf16`	Precision (bf16 default on GPU; `fp32` matches published evals).
`KEV_TEMPERATURE`	(calibrated)	`1.0` returns raw logits.
`KEV_DATE_FACTS`	off	`1` appends day-count sentences for dates in the state.
`KEV_API_KEY`	(none)	If set, `/v1/*` requires `Authorization: Bearer <key>`.

Example: KEV_MODEL=jaredpalmer/kev-0.8b docker compose up -d

Notes

The server is bound to 0.0.0.0:8008 on the host (reachable from other machines on the network). No auth by default; set KEV_API_KEY before exposing it beyond a trusted network.
First docker compose up downloads ~8 GB of weights; subsequent starts use the kev-hf volume.
GPU is used automatically via the nvidia runtime (--gpus all equivalent). On a CPU-only host, build with KEV_BACKEND=cpu docker compose build first.
This POC intentionally uses the official kev.serve runtime: Kev's pointer head + the /v1/systemone contract are not servable by vLLM or the SystemOne llama.cpp fork.

dashaun/kev-poc

README