the call. speech-core is the ear and mouth. a headed omp TUI is the brain. this repo is the phone line between them.
you speak → one committed user turn → one spoken reply you can hear.
law: CHARTER.md. one live audio path at a time.
you (browser or phone)
│
│ SmallWebRTC /desk or PCM /ws-phone
▼
voicecat (this process)
│ 16 kHz PCM in
▼
speech-core VAD / ASR / turn close
│ transcript_committed
▼
CollabGuestService "desk" write-guest on omp collab
▼
headed omp TUI (the only jsonl writer)
│ assistant text as it lands
▼
speech-out leftover first sentence = speak, rest = append
│ progressive PCM
▼
voicecat handset desk speakers or phone call route
this process does not spawn omp. it joins a collab write-link file. missing file → fail closed (collab_link_missing).
| piece | repo | owns | does not own |
|---|---|---|---|
| call | this checkout | transport, PCM hop, collab guest, hear | VAD, ASR, TTS weights, session history |
| ear + mouth | sibling speech-core |
speech-in daemon, leftover TTS | omp, /desk |
| brain | an omp world overlay | headed TUI, collab host, autoStart link file | speech daemons, webrtc |
omp collab.autoStart is opt-in (can1357/oh-my-pi#9833). default off. the overlay turns it on and writes the link file.
- start the headed omp TUI in the talker world so autoStart mints a room and writes the link file (default
~/.local/state/talker/collab.link). - speech-core and speech-out must already be up.
- start this process:
cd python
VOICECAT_ENV=$HOME/.config/voicecat/runtime.env \
.venv/bin/sdc-pipecat-webrtcloopback by default: http://127.0.0.1:7860/desk. allow mic, Connect, speak.
phone APK (android/) is a Graphene call-route client on /ws-phone, not a tiny earpiece pipe. pick input device: bluetooth is SCO and fail-closed (bluetooth headset not connected); phone / speaker / system default stay a normal communication route. hang up and reconnect after changing device. connecting phone replaces desk. details: android/README.md.
TUI restart mints a new room. restart this process so it re-reads the link file.
loopback is only reachable on the machine that binds it. for a browser on another host:
- bind the runner to an address that host can route to (
VOICECAT_WEBRTC_BIND_HOST, not127.0.0.1). - if a TLS reverse proxy terminates HTTPS in front of the runner, set
VOICECAT_WEBRTC_PUBLIC_HOSTto that hostname. the proxy is TCP/HTTPS only. - WebRTC RTP still needs a reachable ICE host candidate. set
VOICECAT_WEBRTC_ICE_HOSTto the same routable IPv4 the browser can send UDP to. otherwise the page loads and Connect hangs.
do not put overlay IPs or private hostnames in this repo. they belong in the operator env file.
| env | meaning |
|---|---|
VOICECAT_ENV |
optional KEY=VALUE file (default config/local.env) |
VOICECAT_WEBRTC_BIND_HOST |
listen address (default 127.0.0.1) |
VOICECAT_WEBRTC_ICE_HOST |
host ICE rewrite for remote browsers |
VOICECAT_WEBRTC_PUBLIC_HOST |
hostname printed for a TLS front |
VOICECAT_COLLAB_LINK_FILE |
write-link file; fail closed if missing |
CPA_API_KEY |
inference proxy; required |
SPEECH_CORE_WS_URL / SPEECH_OUT_WS_URL |
ear and mouth websockets |
secrets stay in the operator env file and $XDG_CONFIG_HOME/voicecat/config.json. never commit them.
speech-out protocol:
- first speakable unit of a response →
{type:speak} - later units of the same response →
{type:append}(one pending continuation) - PCM already leaving while later text is still arriving
- a later response waits for the exact prior sink owner to stop, then speaks
mouth ownership is identity-scoped across barge-in, interrupt, and mouth-only filler. a barge writes sidecar JSONL (barge then barge-cut). the TUI paints a custom row. committed user text is the only collab prompt.
empty-sink stop fallback is 280ms. after response finish is on the wire, matching bot_speaking_stop releases ownership so queued finals drain.
ss -ltnshows the bind host and port7860- speech-core and speech-out up
- page Connected, not just loaded
- talker TUI footer reports the local collab relay and a fresh link file
- after a TUI restart, this process was restarted too
| file | use |
|---|---|
| CHARTER.md | law |
| docs/BOARD.md | map |
| docs/SEAM-LATENCY.md | measured lag |
| spec/ | accepted requirements |
| android/README.md | phone PCM client |