The other half of the problem: when there is no right answer to check against

Can a set of physics checks tell you that a flow prediction is wrong, without ever seeing the right answer? The pre-registered kill gate says yes, at 0.906 composite AUC against a threshold of 0.80 written down first. It also says three things it cannot show — and one of its seven numbers turned out to be a property of the machine it ran on rather than of the physics.

Why this project exists next to the other one

COCLEA-SR is the easy case in one specific sense: an exact solution exists. truth/ holds closed forms, src/ holds the solver, the first is forbidden to import the second, and a gate compares them. When the model was wrong, the mathematics said so.

Now take that away. In haemodynamics — blood moving through a heart chamber — there is no closed form for the geometry anyone cares about. A surrogate model predicts a flow field and nobody has the right answer to compare it to. The whole apparatus of the previous project stops applying.

So the question changes shape:

Can you catch a wrong prediction using only what must be true of any answer?

Mass is conserved. Momentum balances. Velocity is zero at a wall. None of those requires knowing the correct field — they are properties any correct field has. Each becomes an oracle: a cheap, independent check that scores a prediction without the answer.

And then the same discipline, one level up

A panel of oracles that agrees with itself is worth nothing. So the panel goes on trial, against a condition registered before the run:

98 predictions, 31.6% of them worse than 5% true error

  AUC (composite)   0.906     kill below 0.80  ->  SURVIVES
  Spearman rho      0.822
  false-accept      2.1%      target <= 2%
  decisions         ACCEPT 48   ESCALATE 18   REJECT 32

Every corruption type that produces more than 5% error is caught:

The single false accept is a Womersley bias case whose true error is 0.050 against a bad-case boundary of 0.050 — the boundary itself, not a miss.

The result that makes it a portfolio rather than a trick

The obvious suspicion is that one good oracle is doing all the work and the others are decoration. It is not what the numbers say. Six of the seven are near a coin flip on their own, and the panel still reaches 0.906:

Remove both A3 and A4 and the remainder still reaches 0.896. Individually weak gates covering different failures beat each of their members — which is the argument for a panel rather than a better single check.

Two of those seven cells had been wrong on this page's ancestor: A5 and A6 were transposed, and A4 read 0.706. Every number in the table is now read out of gates/reports/h0.json by a script that fails the build when it stops matching.

The finding that is worth more than the AUC

On 2026-08-23 the suite was built from a clean clone, on a different machine with a different BLAS. Two things came back, and the first is the sharper one.

The attested report could not have come from the code beside it. eval/h0.py writes runtime: {seconds}; the committed h0.json carried a top-level seconds and no runtime at all. That nesting was introduced by the very commit that existed to make the run reproducible — so the artifact had been regenerated mid-change and never again. A report whose provenance nobody checks is a report, not an attestation.

Then, on a fresh run, the composite AUC, the Spearman coefficient, the decision counts and the false-accept rate came back bit-identical — and A4 alone moved from 0.706 to 0.652.

A4 alone   0.706  →  0.652     49 of 98 measurements differ in their last decimals
                                66 of 98 are exactly 0.0

one uncorrupted case:  1.03e-13  on one machine
                       0.0       on the other
                       → crosses into a 66-wide tie block
                       → drags a rank statistic by 0.054

The composite is untouched because A4 is a HARD gate: it contributes a pass/fail against a threshold far above the noise floor, never its score. But the lesson generalises past this project. A statistic that moves with a library version rather than with the data is not a property of the physics, and the only way to find out which of your numbers are like that is to run them somewhere else. make reproduce is the instrument that was missing; h0.json now records the environment it was produced on, so the next comparison can tell disagrees from was produced somewhere else.

environment recorded inside the artifact

  python   3.13.12
  numpy    2.5.2
  scipy    1.18.1
  machine  x86_64

The report reproduced 1,207 of 1,207 fields bit-identical on a third environment afterwards — the attestation machinery doing its job on hardware it had never seen.

Four more defects, each of which produced a plausible number first

• The quadrature was biased 2.6%. Summing 2πr·dr at every node overshoots the cross-section, because both endpoints are counted in full. That bias is larger than A1's 1% threshold — so the mass oracle would have been measuring the integration rule. Trapezoidal ends fix it to machine precision.
• Womersley had a sign error. The closed form left an O(1) momentum residual and its quasi-steady limit went to zero instead of Poiseuille. Caught by the limit check, not by reading.
• H0 was not reproducible. Perturbations were seeded with hash(), and Python randomises string hashing per process, so every run reported a slightly different AUC. It moved in the fourth decimal and changed no conclusion — which is luck, not a property.
• A10's threshold was invented and fired on a perfect field. The replacement is derived from the momentum equation and holds a flat slack of 2.1 across a sixteen-fold range of sampling rates.

The first two would each have produced a clean H0 number that meant nothing. The third and the fifth would have produced one nobody could reproduce — and the fifth would have looked reproducible while doing it, because the test the third one added compares two processes on one machine and never reads the committed artifact at all.

What this does not show, stated by the project itself

H1 is recorded in the system as open work that cannot proceed, with a null observation and a note explaining why it is stated at all: a scope with nothing red in it reads as a finished one. The activity canvas draws it as a hollow square — held, no verdict — which is a different mark from the one it uses for a step that ran and failed. Getting that distinction wrong in the interface was itself a bug found while building this site.

Why it is on this website

Every number on this page is read out of projects/hemo-verified/gates/reports/h0.json. Twenty-one of them are named individually in check-numbers.py, which fetches the artifact and fails this website's build if the page has stopped carrying one — on every push and nightly, because the artifact lives in another repository and can move without anything here changing. Each of the seven per-oracle cells is named separately, since two of them were wrong once. That is the evidence layer applied to the page describing it.

The check's own limit, since this page is about stating them: it asks whether the page contains a value, so it catches an artifact that moved while the page did not — the failure this project has actually had — and not one mistyped occurrence among several.

And the honest limit of that, the same one the front page now states: the hash chain proves nobody altered these numbers. It does not prove the code that computed them is right. What argues for that here is different and older — the threshold was declared before the run, the oracles never see the answer, and the artifact was re-derived on a machine that had never seen it. Which is how A4 was caught.

The full write-up → Specification Check the numbers yourself Repository