Every prompt and every raw output from testing Ornith 1.5 35B-A3B on one Mac Studio. Nothing here is edited, summarised or cleaned up — the HTML files are exactly what the model wrote, including the broken ones.
The headline build is the 4-bit, 21.6 GB, which runs on a 32 GB Mac. An 8-bit column is included as a control.
Each task was built four times, --no-session every time, so each run starts
from zero context and the only variable is the label.
| File | Quant | Thinking | Prompt |
|---|---|---|---|
4bit-nothink.html |
4-bit | off | prompt.md |
4bit-thinking.html |
4-bit | on | prompt.md |
4bit-thinking-upgraded.html |
4-bit | on | prompt-upgraded.md |
8bit-thinking.html |
8-bit | on | prompt.md |
The three basic runs share one byte-identical prompt. The upgraded prompt is a different file: it adds three forced steps — write a plan, build it, then prove it runs.
The thinking-off runs exist because the earlier Qwen and DeepSeek baselines were run with thinking off, and comparing against them any other way would be unfair.
| Task | 4bit-nothink | 4bit-thinking | 4bit-upgraded | 8bit-thinking |
|---|---|---|---|---|
| Tower defence | works | fails | works | works |
| Spreadsheet | works | works | works | works |
| FPS (three.js) | fails | fails | fails | fails |
All four spreadsheets pass a real formula battery — operator precedence,
parentheses, left-associative subtraction, cell references, SUM, AVERAGE.
8 of 8 each.
"Works" means it was loaded in a browser, the button that starts it was clicked, the state moved and the console was clean. Not that it parsed.
Not one is a design failure. All twelve files contain a plausible, well-structured version of what was asked for. What breaks is a character, or an ordering mistake:
| Build | Cause |
|---|---|
Tower 4bit-thinking |
between: true at init inverts the guard in startWave() — the button is dead forever, and nothing is logged |
FPS 4bit-nothink |
function loop() declared twice |
FPS 4bit-thinking |
new THREE CapsuleGeometry — a missing dot |
FPS 4bit-thinking-upgraded |
<!-- comment --> inside a JS module |
FPS 8bit-thinking |
obstacles used nine lines before it is declared |
The FPS folder has errors.md with the exact lines,
plus 4bit-thinking-PATCHED.html — that file with the single missing dot
restored and nothing else changed. It is a complete, working game.
Thinking. The tower failed with it on and passed with it off.
The upgraded prompt. It asks the model to prove the build runs. The model has no browser, so on the FPS task "prove it runs" became "write a QA file asserting it runs" — while the file had a syntax error in it. A self-check is worth exactly what the checker can observe.
8-bit did not fix the FPS either, and its failure is the most instructive on the whole project: it parses perfectly, renders a complete HUD, sizes its canvas correctly, and the console is clean. Only the 3D scene is missing. Nothing short of executing the page finds it.
| Task | 4bit-nothink | 4bit-thinking | 4bit-upgraded | 8bit-thinking |
|---|---|---|---|---|
| Tower defence | 1m 36s | 2m 27s | 16m 43s | 1m 46s |
| Spreadsheet | 16m 31s | 18m 50s | 21m 23s | 7m 29s |
| FPS (three.js) | 2m 36s | 1m 46s | 25m 29s | 2m 03s |
The 8-bit spreadsheet finished in under half the 4-bit's time, which is worth a second look before assuming 4-bit is the fast one.
01-invoices — three invoices of increasing density, every line
item extracted and checked. 129 of 132 values correct. Open
01-invoices/results.html.
00-setup/ the rig, the model, why not the official MLX build
01-invoices/ invoice images, ground truth, rendered results
02-tower-defence/ 2 prompts, 4 outputs
03-spreadsheet/ 2 prompts, 4 outputs
04-fps-threejs/ 2 prompts, 4 outputs, the failure write-up, 1 patched file
Open any .html straight from disk. The three.js files fetch three.js from a
CDN, so those need a network connection; everything else is self-contained.
