Two brains at different clock speeds
A vision-language model is a good planner and a hopeless controller. It thinks at about one hertz; a robot that wants to not hit things needs to react twenty times faster.
So the split is explicit. The slow brain looks through the camera, reasons about the scene
and emits intent. A reactive controller runs at 20 Hz and turns that intent into bytecode over
UDP to an ESP32-S3-CAM. The robot exposes itself as a callable cartridge —
robot.navigate, observe, describe, speak,
listen, stop — which makes it, structurally, a device driver for a
language model.
Navigation is camera-only visual servoing. No lidar, no map.
What's proven
This is the deepest codebase here: 118 commits, 302 files, 72 TypeScript sources, and 44 test files carrying roughly 768 cases across cortex, memory, dream, navigation and integration. Plus four ESP32 firmware projects, a bill of materials, printable STLs and six MuJoCo scenes.
And none of it is running in CI. There are no workflows — 768 tests that nothing executes automatically. The repo has also been untouched since May.
There is nothing to look at. For a robotics project this is the real problem: the only images in the tree are four test fixtures. No video, no GIF, no screenshot of the thing working. A 2D viewer and MuJoCo scenes exist, so a demo is recordable — it has not been recorded.