core/docs/decisions/ADR-0131.4-composite-math-gate.md

ADR-0131.4 — Composite Math-Expert Promotion Gate (wired)

Status: Accepted Date: 2026-05-23 Author: CORE main agent (Opus 4.7) Depends on: ADR-0131 (composite gate framing), ADR-0131.1 (B1 substrate), ADR-0131.1.S (B1 sealed holdout), ADR-0131.2 (B2 substrate), ADR-0131.3 (B3 substrate), ADR-0131.G.0 (probe substrate) Parent: ADR-0131 Reserved follow-up: the full ADR-0120 contract (9 ADR-0114a obligations not implemented yet for mathematics_logic).

---

Context

ADR-0131 introduced the composite math-expert promotion contract:

The mathematics_logic domain expert promotion contract is revised. The ADR-0120 single-benchmark check (correct_rate ≥ 0.60 on GSM8K) is replaced by a composite requirement: B1 ≥ 0.95 AND B2 ≥ 0.95 AND B3 ≥ 0.95, each with wrong == 0. GSM8K is retained as a stress-test lane that the math expert runs but is not gated on — reported as honest disclosure.

ADR-0131's implementation plan (sub-phase 0131.4) named formation/ratify.py + formation/promote.py as the wire-up points. That was a misidentification: those modules govern the SPECULATIVE → COHERENT bridge for individual teaching examples (ADR-0021), not domain-tier expert promotion. The correct site is core/capability/, where expert_demo.py (the audit-passed gate) already lives and where ADR-0120 reserves expert_promotion.py.

A full ADR-0120 implementation requires substrate for 10 ADR-0114a obligations (sealed holdout, OOD ratio, replay determinism, typed refusal + wrong == 0, perturbation, depth curve, frontier comparison, adversarial, byte-equal lane runner, operation provenance via pack). For mathematics_logic, only 5 of those 10 obligations have substrate landed today:

• #1 sealed holdout (ADR-0131.1.S — B1 sealed)
• #3 replay-equal trace (B1/B2/B3 runners emit trace_hash)
• #4 typed refusal + wrong == 0 (the load-bearing invariant every G. and B-lane has been preserving)
• #7 frontier-baseline comparison (ADR-0131.1.F)
• #9 determinism (every B-lane report is byte-equal across runs)

The other 5 (#2 OOD ratio, #5 perturbation, #6 depth curve, #8 adversarial, #10 operation-provenance-via-pack) need domain-specific substrate that isn't built yet. Implementing those is sequencing-wise after ADR-0131.4, not bundled with it.

Decision

Implement only the ADR-0131-specific revision as a focused module: the composite B1+B2+B3 evaluator. Structure it so a future core/capability/expert_promotion.py implementing the full ADR-0120 contract can consume it as the math-specific substitute for the single-lane coverage check.

What this ADR wires

core/capability/composite_math_gate.py:

def evaluate_composite_math_gate(
    *, b1_public_path, b1_sealed_path, b2_path, b3_path, gsm8k_probe_path,
) -> CompositeMathGateVerdict

• Reads each benchmark's already-committed report.json (no I/O beyond that; no recomputation of lane verdicts).
• Handles the heterogeneous report shapes (B1/B2 use counts: {correct, wrong, refused}; B3 uses metrics: {correct, wrong, cases_total, correct_rate}). Refuses cleanly if the shape is neither.
• Applies the pinned thresholds: correct_rate ≥ 0.95 AND wrong == 0 per benchmark.
• Composes per-benchmark verdicts → composite verdict.
• Computes a reproducible SHA-256 claim digest over the canonical evidence bundle (per ADR-0120 "Signed expert_claims entry with reproducible digest" requirement).
• Emits GSM8K honest-disclosure (admission, wrong, refused, substrate) but does NOT use it in the gate.

CLI wiring at core capability math-expert-gate (added to core/cli.py). Writes evals/math_expert_claims/v1/expert_claims_math_v1.json (unsigned). Reviewer signature via ADR-0092 reviewer-registry is reserved for the broader ADR-0120 wire-up.

Empirical verdict

Run against current main (post-PR #182/#183/#184/#185, pre-G.1):

composite_gate_passed: True
claim_digest:          2bfc7f6c5b06a4c5befef3a9a2629a023518ae97490f6ddf092c1852c966c275

             B1_public  passed=True  correct=185/185  wrong=0  rate=1.0000
             B1_sealed  passed=True  correct=14/14    wrong=0  rate=1.0000
    B2_teaching_corpus  passed=True  correct=40/40    wrong=0  rate=1.0000
    B3_bounded_grammar  passed=True  correct=50/50    wrong=0  rate=1.0000
GSM8K honest disclosure: admission=0/50, wrong=0, substrate=candidate_graph

The math expert is gate-passing under ADR-0131's revised composite contract. The bet ADR-0131 placed — that the architecture's structural strengths align with three benchmarks that measure those strengths — has paid off on first evaluation.

This does NOT mean the full ADR-0120 contract passes (5 of 10 obligations still need substrate). It means the math-specific revision portion of the contract passes today, and the architecturally-aligned bet was correct.

What this does NOT do

• Does NOT implement the broader ADR-0120 10-obligation contract. That requires substrate for 5 missing obligations (OOD ratio, perturbation, depth curve, adversarial, operation-provenance).
• Does NOT promote mathematics_logic to expert ledger status. Ledger promotion is a separate ADR that consumes this gate's output AND the broader ADR-0120 obligations.
• Does NOT sign the expert_claims artifact. Reviewer signature via ADR-0092 registry is reserved for the full wire-up.
• Does NOT touch formation/ratify.py or formation/promote.py (the teaching-example-tier modules ADR-0131's plan misidentified).
• Does NOT recompute the lane verdicts. The committed B-lane reports are the gate's input contract; the lane runners own their verdicts and the gate trusts them. If a reviewer suspects a lane report is stale, re-run the lane.

Trust boundary

• Reads only:
  • evals/math_symbolic_equivalence/v1/report.json + sealed_report.json
  • evals/math_teaching_corpus/v1/report.json
  • evals/math_bounded_grammar/v1/report.json
  • evals/gsm8k_math/train_sample/v1/train_sample_coverage_report.json (disclosure-only)
• Writes only: the path passed to emit_expert_claims_artifact (default: evals/math_expert_claims/v1/expert_claims_math_v1.json).
• No dynamic imports, no shell passthrough, no network.
• Pure function over already-committed JSON; deterministic (verified by test_claim_digest_reproducible and test_artifact_emission_byte_equal).

Tests

tests/test_adr_0131_4_composite_math_gate.py — 12 tests:

• Threshold values are pinned (changing requires a new ADR)
• Both lane-report shapes parse correctly
• Gate passes iff all 4 benchmarks pass
• Gate refuses cleanly on threshold miss, on wrong-count > 0, on missing report
• GSM8K disclosure handled (present, missing, never gates)
• Claim digest reproducible (deterministic)
• Artifact emission byte-equal across calls
• Snapshot test: committed main state satisfies the composite gate

12/12 pass in 0.22s.

CLAUDE.md PR-checklist

• Capability added: wires the ADR-0131-specific composite benchmark evaluator at the right architectural layer (core/capability/); makes the math-expert promotion verdict computable, reproducible, and CLI-callable.
• Invariant proving field validity: every B-lane's wrong == 0 is preserved (the gate gates on it); GSM8K probe admitted_wrong == 0 preserved.
• CLI/eval proving the lane: python3 -m core.cli capability math-expert-gate + pytest tests/test_adr_0131_4_composite_math_gate.py.
• Avoided hidden normalization / stochastic / approximate / unreviewed mutation: Yes. Pure function over committed JSON.
• Trust boundary: read-only inputs, single deterministic write to a documented artifact path.