Evolving Agents Labs

Evolving Agents Labs/Experiments/qa

qa

What if your test suite told you what it had quietly stopped checking?

Prototype Jun 2026
A column of checks with one gone, and an arrow pointing at the gap

Coverage measures the wrong direction

Coverage tools tell you which lines ran. Nothing tells you which checks you used to make and no longer do. Assertions get deleted during a refactor, or quietly weakened to make a flaky test green, and no tool objects — the suite still passes, faster than before.

Here every assertion is fingerprinted and diffed against a stored baseline. A check that vanished between runs comes back as a finding with a name.

Exploration is expensive; regression should not be

An agent driving a real browser finds things a fixed script cannot. It is also slow, non-deterministic and costs tokens on every run — which makes it a bad regression suite.

So the passing paths get crystallized into deterministic Playwright scripts that run in CI for free. Crystallization is partial on purpose: steps that passed become real tests, steps that failed become explicit test.skip() rather than silence. When a frozen test later breaks, it falls back to exploration.

This is the same fluid→crystallized move as agentvcs, pointed at browser tests instead of agent evolution.

What's proven

v0.7.0, with thirteen commits of genuine product progression — Playwright-MCP, then Chrome-first plus crystallization, then static analysis, then baseline tracking, then persona simulation, then timestamped run folders with issue files.

The honest gap: this is 8,000 lines of prompt and agent markdown with no tests and no CI. For a prompt-defined plugin the markdown is the artifact, but nothing verifies it still works — a Claude Code or Chrome extension change breaks it silently. The Chrome-extension dependency also narrows who can run it.

← skillos  ·  skillos_robot →