Experiments toward a universal robot driver — an AI that can pilot almost any robot regardless of how it's built — running on a ~$20 ESP32-CAM toy car.
tools/dashboard.py observability UI mid-task: the raw 320×240 camera (left),
the Depth-Anything-V2 free-space heatmap with the look-ahead band and floor
reference (center), and the live VLM decision log with per-zone openness and
on-rug confidence (right).
Background & motivation (RU): «Отпускной пост про DIY-роботов».
One model isn't enough. Like a person — Kahneman's Thinking, Fast and Slow — the robot needs a fast reflexive system and a slow, deliberate one, and here a third system that watches both and improves them over time. The bet: just as large language models reshaped NLP, a general "robot driver" will do the same for robotics — and the path there is composing fast / slow / self-improving layers rather than training one monolithic end-to-end policy.
-
System 1 — fast & reflexive —
tools/reflex_drive.py+tools/depth_perception.py. Depth-Anything-V2 splits each frame into left / center / right, scores where it's most open, and steers in real time. Dumb and fast: it owns the wheels and close-range obstacle avoidance. The same depth signal acts as the emergency bumper inside the ER driver below. -
System 2 — slow & smart —
tools/er_drive.py, the primary mission driver: a visual-action loop on Gemini Robotics-ER 1.6. ER looks through the camera and emits one bounded parametric command at a time (action / duration / turn-degrees / trim + a target point); the driver executes it, measures what actually happened from the camera, feeds that back, and self-calibrates. Local Qwen2.5-VL / FastVLM remain as offline alternatives insidereflex_drive.py/vlm_local.py. Shared primitives:tools/carlib.py. -
System 3 — self-improvement (the ralph loop) —
tools/evolve_loop.sh+tools/self_improve_loop.sh. The car drives a bounded episode, then a coding agent (Claude Code or Codex) reads the episode's logs and annotated frames plus the persistentMISSION.md/PROGRESS.mdstate, and may rewrite the prompts and logic of Systems 1–2 — a psychotherapist for the robot, fixing deep mechanisms rather than momentary decisions. Then the next episode starts with the new code. It is bounded by a file allowlist, before/after source-hash checks, per-iteration backups with automatic rollback on red tests, and a unittest suite it is not allowed to edit.
See docs/self_improvement.md for the harness and run commands.
(tools/gemini_drive.py, tools/seek.py, tools/vlm_drive.py are earlier
single-loop drivers kept as reference.)
A cheap ESP32-CAM toy car (camera + WiFi + I²C motor driver), ~1800 ₽ / ~$20.
Firmware lives in firmware/VideoCar/; see firmware/README.md
for the build/flash toolchain and the HTTP control API.
