bug: diagnose whitespace-only LLM responses from third-party relays

#815 · open · 2 comments

View on GitHub ↗

ZhulongNT

## Summary GenericAgent can receive a syntactically valid but semantically empty completion: for example, the response payload contains one text block whose value is a single whitespace character (`" "`). The TUI then shows: ```text · LLM returned an empty response. Retrying... ``` The retry succeeds in the observed case, so this is recoverable, but it is confusing to diagnose. ## Observation - Session trace: `model_responses_858881.txt`, 2026-09-21 16:27:46. - The response was represented as a Python object equivalent to: ```python [{"type": "text", "text": " "}] ``` - There was no meaningful `content` or `thinking`; GA correctly treated it as blank and regenerated. - The relevant guard is `ga.py` / `do_no_tool()` (currently around lines 459–467). It emits `[Warn] LLM returned an empty response. Retrying...` and routes through `_retry_or_exit()`. - The TUI replaces `[Warn]` with `·`, which makes the displayed line look less like an actionable warning. ## Important context: third-party relays / gateways This has been observed when using a **non-official API relay/gateway** rather than the model provider's official endpoint. The relay may return an HTTP-successful response while dropping, truncating, or normalizing streamed content to whitespace. This is an observation, not a claim that every relay is faulty; the same symptom could also originate upstream. However, intermediaries make the failure mode substantially harder to attribute because the request technically succeeds. There is a related lower-level path in `llmcore.py` where a stream produces no output and is retried as `ConnectionError("empty response")`, but the whitespace-only case reaches the agent-level blank-response guard instead. ## Why this deserves a separate issue #201 added the useful retry limit (retry up to three times, then stop). It does not expose enough context to distinguish: 1. a provider-side empty completion, 2. a relay/gateway that returned a malformed or whitespace-only successful response, and 3. a stream that ended before producing any content. ## Suggested improvements 1. Keep the current safe retry behavior. 2. Log a compact, redacted diagnostic for blank responses: session/model name, configured base URL host (no credentials), response block types, text lengths / whitespace-only flag, whether any thinking was present, and retry count. 3. Make the TUI warning retain a visible severity label, e.g. `· [Warn] Empty LLM response; retrying (1/3)`. 4. Optionally classify `whitespace_only`, `no_stream_output`, and `empty_payload` separately in the trace/logs. 5. Documentation: recommend reproducing against the official endpoint before filing provider-specific issues when a third-party relay is in the path. This should make relay-related incidents diagnosable without weakening compatibility with OpenAI-compatible gateways.

Comments

louisss1016

I'd like to take this one. Plan, following the issue's suggested improvements: 1. **Diagnostic at the guard** — in `ga.py` `do_no_tool()`, keep the safe retry path unchanged, and emit a compact redacted diagnostic on blank responses: classification (`whitespace_only` vs `empty_payload`), response block types, content/thinking length and whitespace flag, retry count, model name, and the gateway **host** only (no path, no credentials). 2. **TUI severity label** — in `frontends/tui_v3.py`, stop stripping `[Warn]`/`[Warning]`/`[Error]` into a bare `· `; render them as `· [Warn] ...` so the severity survives. 3. **Tests** — pytest coverage in `frontends/tests/` for the classifier and the TUI prefix rule. The retry behavior itself (up to 3, then stop) stays exactly as it is — this is diagnosability only, no compatibility change for OpenAI-compatible gateways.

louisss1016

PR up: https://github.com/lsdefine/GenericAgent/pull/819 Summary of the implementation: - `ga.py` gains `describe_blank_response()` (kind / sizes / block types / model / gateway host — hostname only, never credentials) and `_url_host()`; `do_no_tool()`'s retry semantics are untouched (3 attempts then exit), only the warning line gains the attempt counter + diagnostic. - `tui_v3.py` splits `_ACTION_RE` so `[Warn]/[Warning]/[Error]` now render as `· [Warn] …` instead of a neutral bullet. - 15 new tests via the repo's usual ast/exec extraction pattern; full suite 287 passed (the 3 failures are pre-existing Windows-env issues: symlink perms ×2 and the release-qualification test that needs a built package). Left out on purpose: the `llmcore.py` `no_stream_output` path (already a distinct `ConnectionError`), and suggestion #5's docs note — that one feels like a maintainer call, happy to add it if you want it in this PR.