Commit graph

1 commit

Author SHA1 Message Date
Shay
29d916ab16 docs(ADR-0131): re-target math expert promotion to architecture-aligned benchmarks (proposed)
Docs-only PR. Follow-up to ADR-0127-0128-RESULTS' Path-B trigger.

Replaces ADR-0120's GSM8K-coverage requirement for the
mathematics_logic expert promotion with a composite gate over
three architecture-aligned benchmarks:

  Benchmark 1: Symbolic equivalence (~500 algebra cases, public
    + sealed split). Primary discriminator — CGA exact-recall.
    Pass: correct_rate >= 0.95, wrong == 0.

  Benchmark 2: CORE-native teaching-corpus eval. Internal
    consistency lane sourced from ratified packs (en_arithmetic_v1
    + en_units_v1 + en_numerics_v1). Grammar matches by
    construction (no paraphrase-variance gap).
    Pass: correct_rate >= 0.95, trace_hash byte-equality, wrong == 0.

  Benchmark 3: Bounded-grammar word-problem set (~150 cases,
    pre-filtered to ADR-0126/0127/0128 parser scope). Demonstrates
    end-to-end pipeline determinism on the in-scope distribution.
    Includes adversarial out-of-grammar probes.
    Pass: correct_rate >= 0.95, wrong == 0.

Three benchmarks (not one) prevent overfitting per ADR-0114a
and force the math expert to demonstrate three distinct
architectural properties (algebraic correctness, internal
consistency, end-to-end pipeline determinism).

GSM8K stays as a stress-test lane the math expert RUNS but is
NOT gated on. Refusal counts reported in expert-claims artifact
as honest disclosure.

Other 12 ADR-0120 expert-promotion checks UNCHANGED.

Status: Proposed. The contract revision (removing GSM8K from
the gate) is a follow-up implementation PR after sub-phases
0131.1-0131.6 land the new benchmark substrate.
2026-05-23 07:49:18 -07:00