Commit graph

4 commits

Author SHA1 Message Date
Shay
f56a0cfdba
Merge pull request #546 from AssetOverflow/docs/reconcile-current-state-2026-06-03
docs: reconcile current-state claims after GSM8K + sensorium progress
2026-06-04 07:14:39 -07:00
Shay
763c46d2f4 fix(gsm8k): disable unsound serving bridges — restore sealed wrong=0 (0/5 -> 0/0)
The FIRST real sealed measurement (operator-decrypted 1,319 held-out GSM8K)
found `0 correct / 5 WRONG` — a wrong=0 breach hidden for weeks because the
working metric was the 50-case train sample the bridges were tuned to. Bisection
isolated it to the product_bridge serving promotion (ADR-0195).

- generate/math_candidate_graph.py: REMOVE both serving promotion bridges
  (product_bridge + goal_residual/ADR-0207 §5 step 2). Serving = main-graph-only.
  Restores sealed 0/0/1319 (verified by bisect: disabling product_bridge -> 0 wrong).
  Production modules remain in generate/derivation/; only serving promotion is
  unwired, until a gate is proven wrong=0 on the SEALED set (never the train sample).
- Honest numbers everywhere: train_sample 7/43/0 -> 4/46/0 (the bridges' "correct"
  was train-overfit). report.json + coverage probe regenerated. 7 ADR test lanes
  de-pinned from the inflated count. corpus: cv-0005 (R4) reverts to refuse; cv-0020
  (a "baseline control" that solved ONLY via product_bridge) reclassified.
- docs/claims_ledger.md: dated wrong=0-breach-and-remediation note + the rule:
  the train_sample number had ZERO predictive validity for the exam; never the score.
- docs/analysis/gsm8k-lift-program-strategy: the program to actually move the 1,319.

NOTE the exit gate stays `correct>=10 AND wrong==0` — refusing-everything is an
explicit FAIL, not a wrong=0 pass; serving still commits (main graph). Verified:
broad regression 848 passed, smoke 73 passed.
2026-06-04 01:55:05 -07:00
Shay
e7392f118e docs: reconcile claims ledger with current state 2026-06-03 22:41:19 -07:00
Shay
96b6c5e4b2 docs(claims): ADR-0200 + claims ledger — single source of truth (additive)
Add docs/claims_ledger.md (every public claim -> in-repo evidence, with the
four GSM8K numbers separated and labeled) and ADR-0200 recording the
mathematics_logic expert fail-closed revert as designed behavior (proven
single-source evidence-drift; determinism intact).

Additive only: no existing claim, eval gate, or test is touched. The
review-gated reconciliation diffs (reviewers.yaml quarantine note, ADR-0120
header note, signed-JSON regen, README narrative, 3 test reconciliations) are
listed in ADR-0200 section 4 and await operator ratification.
2026-06-02 09:35:44 -07:00