core/docs/analysis
Shay c2e74fe2fd docs: the honest real-GSM8K capability measurement (0% sound; composer 17% wrong on held-out)
The result that should have been visible for weeks. Measured on the new holdout_dev
lane (500 real held-out cases) + sealed test:
- real capability 0% (train_sample's 4 correct generalize to 0/500 held-out)
- resolve_pooled (the composition substrate) commits 17.4% WRONG on held-out
- NO structural gate (step/pool/op) separates its 2 correct from 87 wrong -> the
  composer cannot tell right readings from wrong; gating it is overfitting
- the only wrong=0 policy on real GSM8K is refusal (current serving 0/0/500)

Real lift needs a verifiable reading mechanism, not composer tuning. Non-serving (docs).
2026-06-04 02:42:14 -07:00
..
composition-capability-scope.md docs: mark ADR-0174 Phase 5a shipped in composition-capability-scope 2026-06-03 16:23:53 -07:00
composition-wall-execution-plan-2026-06-03.md feat(r4): flip cv-0005 to serving — train_sample 6/44/0 -> 7/43/0 (ADR-0207 §5 step 2) 2026-06-03 22:20:12 -07:00
comprehension-primitive-inventory.md Add canonical composition analysis docs 2026-06-03 15:35:58 -07:00
extraction-richness-audit-2026-06-03.md docs: ratify ADR-0207 — GSM8K comprehension/composition substrate 2026-06-03 19:42:47 -07:00
gsm8k-lift-program-strategy-2026-06-04.md fix(gsm8k): disable unsound serving bridges — restore sealed wrong=0 (0/5 -> 0/0) 2026-06-04 01:55:05 -07:00
question-layer-gap-survey.md Harden no-reference n-times comparative guard 2026-06-03 15:22:42 -07:00
real-gsm8k-capability-measurement-2026-06-04.md docs: the honest real-GSM8K capability measurement (0% sound; composer 17% wrong on held-out) 2026-06-04 02:42:14 -07:00
solver-operation-coverage.md Harden no-reference n-times comparative guard 2026-06-03 15:22:42 -07:00
wiring-promotion-gate-brief-2026-06-03.md docs: composition-wall execution scope (ADR-0207 §5 step 2) 2026-06-03 21:14:02 -07:00