A multimodal AI assistant that runs entirely on your machine via @qvac/sdk. No API keys, no cloud, no network at inference time — only the first model download touches the internet.

Chat · Vision · RAG · Transcription · Text-to-speech · Image generation · Music generation

npm install
npm start

Open http://localhost:8787.

That is the whole setup. Models download automatically the first time each capability is used, and are cached in ~/.qvac for every later run.

Before a live demo, pre-warm the models. The first use of each capability blocks on a download — up to ~5 GB for image generation. Run npm run smoke once on the demo machine and everything afterwards is instant.

Verified on macOS 15 / Apple M3 Max. npm install pulls ~4.8 GB of prebuilt native engines (llama.cpp, whisper.cpp, stable-diffusion.cpp, ONNX TTS, audiogen, Parakeet) and takes several minutes — that is expected, not a hang.

Run one capability at a time while debugging:

node smoke.js chat | vision | tts | asr | rag | image | music

Pick with QVAC_TIER (default quality):

npm start                 # quality — better answers and images
QVAC_TIER=fast npm start  # fast — half the disk, quicker first run

In the quality tier Qwen3.5 is multimodal, so chat and vision share one loaded model rather than holding two sets of weights. The sidebar shows Vision as shared.

Override any single model with an env var (the value must be a constant exported by @qvac/sdk; an unknown name fails at startup with a clear error):

QVAC_CHAT_MODEL=QWEN3_5_9B_MULTIMODAL_Q4_K_M npm start

QVAC_CHAT_MODEL · QVAC_VISION_MODEL · QVAC_VISION_MMPROJ · QVAC_EMBED_MODEL · QVAC_ASR_MODEL · QVAC_TTS_MODEL · QVAC_TTS_VOICE · QVAC_IMAGE_MODEL · QVAC_IMAGE_TEXTENC · QVAC_IMAGE_VAE · QVAC_MUSIC_*

The sidebar shows every model's state (idle / loading with a progress bar / ready) and a live activity log, so an audience can see work actually happening on the machine.

Chat — type and press Enter.

• 📎 attach an image to ask about it (switches the turn to the vision model)
• 🎙️ record from the microphone; audio is transcribed and dropped into the box
• Read replies aloud speaks every answer
• Ground answers in my documents turns on RAG

Knowledge — paste or upload .txt / .md, press Ingest. Then enable grounding in Chat: retrieved passages appear above the answer with their source filename and [1]-style citations. Search tests retrieval directly and shows similarity scores.

Image — a prompt, a size and a step count. 20 steps at 512×512 is a good default; fewer is faster and rougher.

Music — a caption, optional lyrics (leave blank for instrumental), and a duration. Output is a playable, downloadable WAV.

Free heavy models releases image and music from memory (~9 GB) while keeping chat, vision, RAG, transcription and speech resident.

npm run smoke

Runs every capability against the real engines and writes what it produced to data/smoke/ — a PNG, two WAVs, and transcripts — so you can look at the output rather than trust a green tick. The asr test is a genuine round-trip: it synthesises speech with the TTS model and transcribes it back, so both engines must really run for it to pass.

npm start            # in one terminal
npm run smoke:http   # in another

Covers static assets, /api/capabilities, the SSE event stream, streaming chat, TTS, RAG ingest/search, grounded chat with citations, and transcription.

Everything the UI does is available over HTTP.

curl -N -X POST http://localhost:8787/api/chat \
  -H 'content-type: application/json' \
  -d '{"history":[{"role":"user","content":"Hello"}]}'

server.js            Express + SSE endpoints
smoke.js             end-to-end capability tests
http-smoke.sh        HTTP surface tests against a running server
qvac.config.json     QVAC plugins, logging, download retry policy
src/models.js        capability -> model wiring (the file to edit)
src/engine.js        lazy model manager + one function per capability
src/wav.js           raw PCM -> .wav
public/              single-page UI (no build step, no framework)

npm install seems stuck. It is downloading ~4.8 GB of native engines. If it fails with ECONNRESET, npm rolls back the whole tree — rerun with npm install --fetch-retries=8 --fetch-timeout=900000.

First request hangs for minutes. It is downloading a model. Watch the sidebar progress bar or the server log.

Out of memory. Click Free heavy models, or use QVAC_TIER=fast.

Reclaiming disk. Models live in ~/.qvac. Deleting it is safe; they re-download on next use.

RAG returns nothing. Ingest documents first. Changing QVAC_EMBED_MODEL or the tier starts a separate knowledge base, so re-ingest after switching — switching back finds the earlier one intact.