Evolving Agents Labs every number checked nightly

Evolving Agents Labs is an open-source lab for agents that learn how a team works — and can show what they learned. The director keeps the team’s way of working as a memory of small, linked statements, plans by walking it, delegates each step to a frontier model or to local models, and learns from every task. lora-kernel serves small local models as trained specialists. agentvcs versions and merges the harness those models run in, and attributes each change in a result to the change that caused it. gene-evidence writes evidence reports on predicted genes in which every sentence cites its source. The idea they share: what an AI system answers, changes or reports should be traceable to what produced it. What is built and what is planned between them: how they connect.

director

Can an agent keep a team’s way of working as memory — and plan, delegate and learn by it, so that each lesson is checked before it is kept?

active · D0–D3 measured  A meta-agent in evolving-agents: walks the operational memory to plan, hands each step to Claude Code or to a local model, learns from each task and from where the user moves work.

Built: the loop and its local interface, learning sealed as agentvcs patches (measured). Not yet: its local lane through lora-kernel’s specialists; no write access by default.

Why it matters: the rules a team writes down are not the ones it follows under pressure. If it works, the way of working is recalled at the moment it applies, and every new lesson is a reviewed, reversible change instead of a line in a file nobody rereads.

lora-kernel

Can one small local model, given a trained specialist per job and a library of notes it has learned to navigate, do an organisation’s routine work — and hand the rest to a frontier model?

active · flagship  An operating layer for specialized local AI agents: one resident model, a LoRA per job, facts in notes a person can edit, a router that abstains.

Built: specialists that beat the bare base on their job, a router that serves no foreign text locally, edits answered without retraining — measured. Not yet: no installable package, no real traffic.

Why it matters: if it works, an organisation’s routine AI work is served from one small local GPU — private, cheap to run, its facts in notes a person can read and edit — and only the rest is sent to a frontier model. The saving is not measured yet.

agentvcs

When an agent’s harness changes while it runs, can every result still be traced to the change that caused it — and can two diverging harnesses be merged on the evidence?

active · Rust core v0.1  Version control for agent harnesses: every step stamped with the harness version that produced it, patches gated, blame per metric, and a three-way merge whose real conflicts go to Claude Code — its only LLM.

Built: the Rust core, CLI, MCP server and Python SDK; attribution on a real model (below). Not yet: no PyPI package; no real Claude Code merge session recorded in the repository.

Why it matters: harnesses now change while agents run — a prompt, a model, a router. If it works, you know which change moved which result, and two harnesses that diverged are merged on evidence rather than by guess. Attribution has run on one real model.

gene-evidence

Can every sentence of a report on a predicted gene be traced to the tool output it rests on?

prototype · stage 1  Auditable evidence reports for gene predictions the reference annotation does not have: an evidence graph of tool outputs, and a report that cites it in every sentence.

Built: stage 1, deterministic, no language model; a development run on rat. Not yet: a model that drafts the report. Known limitation: it explains a candidate; it does not pick which to validate first.

Why it matters: a wet-lab experiment on a candidate gene costs real money. If it works, every claim in the evidence report can be checked against its source before that money is spent. The checkable report is built; choosing what to test is not.

A workshop room. In front, a chart table with a map of linked index cards and a route traced in red; two brass tubes leave it, one to a window and a distant domed building, one to a small machine nearby. Behind, three benches: a card-catalogue library, a machine with a sealed ledger, an evidence table with threaded cards. Solid cords join the chart table to the ledger and the library to the ledger; dashed lines mark the links still planned.
The director’s chart table in front, the three benches behind: lora-kernel, agentvcs, gene-evidence. Solid cords exist today; dashed lines are planned.

What they share is one rule: a statement that does not cite its source does not ship. gene-evidence’s check-report fails a sentence whose citation does not resolve; lora-kernel’s endpoint withholds an answer whose citation fails the referee’s check.

The director

Plan by the team’s memory; learn by gated patches.

The memory is a small wiki of the team’s practices: pages of atomic statements (when this situation, do this, because of that incident), each linking to the ones that apply next. An index names the situations where a walk starts. The director walks it before every plan and cites what it followed; after a task it proposes edits, and a person accepts or rejects them. Code: packages/director.

Measured

Against our own previous way of working, never a frontier model.

ran  Planning our way, 14 real situations, blind judge: today’s session 0.589, told to walk its own files 0.768, the director 0.929 (D0).

ran  Learning from tasks: held-out probes of lab-specific lessons from 0.0 to 0.781, the first suite re-checked at 0.964 (D2). Learning the lane: after one move, public-facing drafts went 0 / 6 → 6 / 6 to the frontier while internal text stayed local (D3).

not yet  It costs about eleven times more per plan than today’s session; the cases come from the lab’s own history, not from future work.

Three bar charts. D0, planning the team's way on 14 situations: today's session 0.589, today's session told to walk its files 0.768, the director 0.929. D2, learning from tasks on 8 held-out probes: before 0.0, after learning 0.781, and D0 re-checked afterwards 0.964. D3, learning the lane: public-facing drafts sent to the frontier before 0.0, after one move 1.0, and the internal control kept local 1.0.
D0, D2 and D3, drawn from the briefs’ numbers.
An open index book with one line underlined; a red-pencil route leaves it and steps across five index cards, leaving each from its highlighted line, to a plan of three items, each tagged back to the card it came from.
It plans by walking the memory: each step of the plan is tagged to the statement it follows.
A finished work order with a correction pinned to it; a new index card drafted at a desk; the card checked at a small brass gate; a ledger line closed with a fresh seal; the card filed in a cabinet drawer; a dashed arc back to the start.
It learns from a task: a correction becomes a statement, checked at a gate, sealed in the ledger, filed.
A dispatch board with two columns of card slots: the left bordered in rust under a window with a distant domed building, the right in green under a small local machine. One card moves mid-air from the right column to the left; a lever below is thrown; the card beside its empty slot stays put.
It learns where work runs: a task the user moved to the frontier teaches it where the next like it goes.

Each step names its measurement before it is built, against the version it replaces; a flat result is published as flat. Order, not dates. The full table: ROADMAP.md.

A railway survey map: seven stations along one line. The first three are drawn and coloured, joined by solid green track, each with a wax seal; the next four are pencil outlines joined by dashed ochre track. A surveyor's level stands where the built track ends.
  1. Built: plan by walking the memory (D0), the loop and its interface (D1), learn from a task (D2), learn where work runs (D3), an Accept guarded by D0.
  2. Your own director in ten minutes — director.compile on your CLAUDE.md and memory; judged by a first user’s compile and the time and cost from a clean clone.
  3. The local lane through lora-kernel — a specialist serves its domain on your machine, the rest abstains to the frontier; judged by the share served locally and no abstained task lost.
  4. Cheaper walks — today about eleven times today’s session per plan; judged by D0 held within 0.03 at three times the cost or less.
  5. Inside Claude Code — the walk in the session you already use; judged by D0 run there.
  6. A team’s memory — lessons shared and merged with their evidence; judged by two members’ lessons merged without losing either’s probes.
  7. Held-out from the future — cases from work done after the memory was compiled. Bring yours: a situation where your agent ignored a rule you had written down.

The flagship · lora-kernel · Apache 2.0

The LoRA is not the textbook. It is the specialist who knows how to use the library.

One GPU, one small resident model, and a shelf of QLoRA adapters — each an expert defined by its corpus. An OpenAI-compatible API serves each request with the matching expert and sends everything else to a frontier model. Facts live in short notes a person can read and correct; the adapter holds the habit of navigating them.

What is measured, and what is not → The architecture article Repository

A reader at a desk before a wall of card-catalogue drawers labelled Procedures and Encyclopedia; a small radar dish on the desk lights three drawers; through an open door, a domed building on a hill labelled frontier.
The specialist, the library, and through the door the frontier for everything else.

agentvcs

Change your harness while it runs and still know what worked.

Every step of a run is stamped with the harness version that produced it, so a patch applied mid-run can be gated, attributed with blame, rolled back or resumed from a checkpoint. Diverged harnesses merge dimension by dimension; each real conflict goes, with its evidence, to Claude Code, agentvcs’s only LLM, and the resolution is checked before it is recorded.

Repository → The July 2026 write-up (earlier Python version)

Measured, and not

A Rust core, CLI, MCP server, Python SDK and a conformance suite.

ran  On Qwen2.5-1.5B on llama.cpp, a patch applied mid-run passed a paired gate with a recall gain of +0.444, and blame attributed +0.442 to exactly one patch — the one applied (RUN_REAL.md).

ran  bisect re-executes each candidate from the run’s checkpoint: on a toy run where a regression landed next to a harmless patch, blame could only credit the pair; bisect named the regression (toy pipeline).

not built  No PyPI package; no real Claude Code merge session recorded; no remotes or sync.

A brass machine on a bench with one freshly replaced gear; a paper ledger unrolls from it, each line sealed in wax, blue before the change and orange after, and a red ribbon ties the new gear to the first orange seal.
Every step stamped with the harness version that produced it.

gene-evidence

Auditable evidence for gene predictions the reference annotation does not have.

Some predicted genes are absent from the species’ reference annotation: missed genes, or artefacts. For each, gene-evidence analyze builds an evidence graph — locus context, retrocopy signature, Swiss-Prot homology, Pfam domains, ORF sanity — each node one tool output with its version and parameters. The report cites nodes in every sentence and check-report fails if one does not. No language model; the rules are in EVIDENCE.md.

Repository → The rat run

The first run, on rat

A development run, not a test of ranking quality.

947 candidates; 578 of 947 (61 %) sit on loci the reference calls pseudogenes, which is why locus context is read before homology. Two runs gave byte-identical graph and report, and every sentence passed the citation check; 35 unit tests. The top 100 are all tier 1 and repeat-rich families fill them, as written down before the run: 32 hit the Smok kinase family, 24 T-cell receptor or immunoglobulin variable segments, 2 retroviral Pol polyproteins (rank 1 among them).

known limitation  On plant genomes, against long-read transcript evidence, the tiers ranked supported candidates worse than the predictor’s own confidence score. Use the report to explain a candidate, not to choose which to validate first (README).

Bar chart of the 947 candidates of the rat development run by what the reference annotates at the locus: pseudogene 578, nothing 285, protein-coding gene on the other strand or in an intron 45, non-coding gene 39.
The rat run by what the reference annotates at each candidate’s locus.
One candidate's section of a gene-evidence report: evidence for, evidence against, context and a suggested validation experiment, each line ending in bracketed graph-node citations.
Rank 1 of the rat run: every line ends in the graph nodes it cites.

Twenty-five more are archived and read-only, listed in the archive. agentvcs is among the projects that passed the ANFAIA Research Lab.

Questions, reproductions that disagree with a number here, and proposals are welcome as issues: lora-kernel · agentvcs · gene-evidence · everything else under github.com/EvolvingAgentsLabs.