Commit graph

1 commit

Author SHA1 Message Date
Shay
4b59f3daf7 feat(ADR-0131.4): composite math-expert promotion gate — wired, evaluated, PASSING
Implements ADR-0131's revision of the ADR-0120 expert-promotion
contract for mathematics_logic: replaces the single-benchmark
GSM8K-coverage check with a composite B1+B2+B3 requirement.

New module core/capability/composite_math_gate.py:
  - evaluate_composite_math_gate(): pure function over already-
    committed B-lane reports; handles heterogeneous report shapes
    (B1/B2 counts vs B3 metrics); applies pinned thresholds
    (correct_rate >= 0.95 AND wrong == 0); composes verdicts.
  - Reproducible SHA-256 claim_digest over canonical evidence bundle.
  - GSM8K honest-disclosure (admission/wrong/refused/substrate)
    embedded in artifact but never gates per ADR-0131.

CLI: core capability math-expert-gate (added to core/cli.py).
Writes evals/math_expert_claims/v1/expert_claims_math_v1.json.

Empirical verdict on current main (post-PR #182/#183/#184/#185):
  composite_gate_passed: True
  B1_public:          185/185 wrong=0 rate=1.0000
  B1_sealed:           14/14  wrong=0 rate=1.0000
  B2_teaching_corpus:  40/40  wrong=0 rate=1.0000
  B3_bounded_grammar:  50/50  wrong=0 rate=1.0000
  GSM8K disclosure:    0/50 admission, wrong=0, substrate=candidate_graph

The math expert is gate-passing under ADR-0131's revised composite
contract. The architectural bet ADR-0131 placed has paid off.

Honest scope-limit: this implements only the ADR-0131-specific
revision (composite benchmark portion). The full ADR-0120 10-
obligation contract still requires substrate for 5 missing
obligations (OOD ratio, perturbation, depth curve, adversarial,
operation-provenance-via-pack). Those are sequencing-wise *after*
ADR-0131.4, not bundled. Reviewer signature via ADR-0092 registry
is also reserved.

Trust boundary: read-only access to 5 committed lane reports;
single deterministic write to the artifact path. No dynamic
imports, no recomputation of lane verdicts.

Tests: 12/12 in tests/test_adr_0131_4_composite_math_gate.py
covering threshold pinning, heterogeneous shape handling, gate
logic (passing + every failure mode), GSM8K honest disclosure
(never gates), determinism (claim_digest + artifact byte-equality),
and a snapshot test confirming current main satisfies the gate.

ADR-0131.4 module note: the parent ADR-0131 plan named
formation/ratify.py + formation/promote.py as the wire-up site —
that was a misidentification (those govern teaching-example
SPECULATIVE→COHERENT bridging per ADR-0021, not domain-tier
promotion). Correct site is core/capability/, where audit-passed
gate already lives.
2026-05-23 15:23:14 -07:00