core/docs/decisions/ADR-0121-mathematics-logic-expert-deferred.md

ADR-0121 — mathematics_logic expert Promotion — Deferred (first attempt)

Status: Accepted (the deferral is the decision) Date: 2026-05-23 Author: CORE agents + reviewers Depends on: ADR-0107 (the same pattern for audit-passed), ADR-0110 (math audit-passed promotion), ADR-0114, ADR-0114a, ADR-0119 (+ all 8 sub-phases), ADR-0120 (first expert promotion contract) Supersedes: none

---

Context

ADR-0120 defined the expert promotion contract: a 13-check gate that composes ADR-0114a's 10 anti-overfitting obligations with three contract-level requirements (audit_passed holds, correct_rate ≥ 0.60 on public AND sealed holdout, signed expert_claims entry with reproducible digest).

This ADR is the first worked attempt at promoting a domain under that contract. Following the doctrine the project has now established twice (ADR-0107 → ADR-0110 for math audit-passed; ADR-0122 → ADR-0124 for systems_software audit-passed), the first attempt may defer honestly on a named architectural blocker. That deferral is the contract working as designed — not a setback.

The candidate is mathematics_logic because:

• It is already at audit-passed (ADR-0110)
• Its lanes carry the most complete substrate (ADR-0119 + 8 sub-phases)
• It has the only sealed external benchmark (real GSM8K test, ADR-0119.7)
• The framing arc started here (ADR-0114 picked it as the first expert-capability target)

---

Attempt

Evaluating each ADR-0120 gate against the live state of main:

ADR-0114a obligations (10 of 10 pass)

#	Obligation	Live measurement	Threshold	Pass?
1	Sealed-holdout discipline	gsm8k_math/holdouts/v1/cases.jsonl.age exists; runner refuses without CORE_HOLDOUT_KEY	encrypted seal in place	✓
2	OOD surface variation ≥ 0.95 of public	OOD/public ratio = 1.00 (150/150 across 3 transform families)	≥ 0.95	✓
3	Replay-equal trace	ADR-0117 verifier passes on every public-split correct outcome	byte-equal replay	✓
4	Typed refusal + wrong == 0	public 150/150 correct, 0 wrong; sealed 0/1319 correct, 0 wrong	wrong == 0 both splits	✓
5	Reasoning-isolation perturbation suite	invariance-preserving 207/207, invariance-breaking 17/17	1.0 each	✓
6	Compositional-depth curve flatness	depth 1–8 all at rate 1.0 on public (perfectly flat)	accuracy(N) ≥ accuracy(d1)·0.95^(N-1)	✓
7	Frontier-baseline comparison	comparison_v1.json with 3 citations dated 2023+; CORE measurement embedded	≥3 citations, dated 2023+	✓
8	Adversarial generation; misparse zero	38 cases × 12 families; correct 5, wrong 0, refused 33	wrong == 0	✓
9	Determinism	runner + solver + verifier + realizer all byte-equal across two runs	byte-equal	✓
10	Operation provenance via pack	every step's pack_lemma_id resolves in en_arithmetic_v1	resolves	✓

All 10 obligations discharged.

ADR-0120 contract-level gates (1 pass, 2 fail)

Gate	Live measurement	Threshold	Pass?
audit_passed_holds	predicates.audit_passed == True (since ADR-0110)	must hold	✓
correct_rate_minimum (public ≥ 0.60)	public correct_rate = 1.0 (150/150)	≥ 0.60	✓
correct_rate_minimum (holdout ≥ 0.60)	sealed-holdout correct_rate = 0.0 (0/1319)	≥ 0.60	✗
signed_expert_claim	no expert_claims entry for mathematics_logic	signed claim required	✗

The substantive blocker is the sealed-holdout correct_rate gate. The signed-claim gate is downstream — no reviewer signs a claim whose underlying measurement doesn't meet the contract.

---

Decision

mathematics_logic is NOT promoted to expert. The contract refuses. The ledger row remains at audit-passed.

The gate's named blocker:

"lane gsm8k_math correct_rate on sealed holdout = 0.0 (0/1319), below contract threshold 0.60."

This is the contract working as designed. ADR-0120 §"Threshold rationale" deliberately set the floor above current measurement specifically so the first promotion attempt would defer honestly.

---

What this proves (and what it doesn't)

Proves

• The substrate is complete enough that the gate has all the inputs it needs to make a decision — there's no "missing obligation" excuse. All 10 ADR-0114a obligations are mechanically verifiable on disk. The deferral is a substantive judgment about capability, not a process gap.
• CORE's wrong-zero discipline holds against a real external benchmark. On 1,319 real GSM8K test problems, the system refuses every one it can't grammar-handle without confabulating on a single one. That property is more rare and more valuable than the correct_rate gate.
• The contract is genuinely falsifiable. A weaker gate would have rubber-stamped the promotion on the strength of the obligations alone. ADR-0120's floor refuses honestly.

Does NOT prove

• That CORE will never reach expert on math. It explicitly will, once the parser grammar covers enough GSM8K-style constructions to lift the sealed-holdout correct_rate above 0.60.
• That the 0.60 floor is the right number forever. ADR-0120 §"Open candidate directions" notes that the floor may be raised in a future amendment as CORE catches up.
• Anything about the other audit-passed domains (physics, systems_software). They have their own promotion ADRs ahead of them and their own substrate gaps.

---

What would unlock the promotion

A multi-ADR parser-expansion arc lifting the sealed-GSM8K correct_rate from 0.0 to ≥ 0.60. Each ADR in the arc adds one construction class to the parser grammar:

1. Rate / per-unit reasoning ("Each item costs $2; X buys 4")
2. Comparison phrasing ("X has 3 more than Y")
3. Percentage / fraction ("Half the apples", "20% of N")
4. Time-modal / temporal ("How long does it take?")
5. Multi-step conditional ("If X then ...")
6. Set / collection language ("The students who passed ...")
7. Aggregation / summation ("In total, after N steps ...")
8. Unit conversion ("How many minutes in an hour ...")

Each ADR ships:

• A grammar extension (parser + solver + verifier + realizer updates)
• A re-measurement on the sealed holdout (single number: new correct_rate)
• An ADR-0118a OOD re-measurement (no surface-feature regression)
• An ADR-0125 perturbation re-measurement (no invariance regression)
• An ADR-0119.5 adversarial re-measurement (no new misparses)

The honest-fitting discipline: every expansion is graded on ADR-0114a #2 / #5 / #8 BEFORE the correct_rate lift counts. A correct_rate lift accompanied by an OOD regression IS a regression, not progress. This is the structural defense against silent overfitting to the sealed holdout.

Estimated number of expansion ADRs to reach 0.60: 4–8. Honest acknowledgment: nobody knows in advance. The arc lifts the number, each lift gets reviewed against the anti-overfit obligations, eventually crosses 0.60 — or doesn't, in which case the contract remains refusing and we update the ADR-0114 roadmap with the learned constraint.

---

Invariants

adr_0121_math_remains_at_audit_passed

ledger_report() reports mathematics_logic with status == "audit-passed" and predicates.audit_passed == True. The hypothetical predicates.expert either does not exist yet (until ADR-0120a implementation lands) or is False. Tested by tests/test_adr_0121_math_expert_deferred.py.

adr_0121_sealed_correct_rate_below_floor

Running the lane runner against the sealed GSM8K test (decrypted with CORE_HOLDOUT_KEY) yields correct_rate < 0.60. The specific measurement today is 0.0 (0/1319). The test pins "< 0.60" rather than the literal 0.0 so future parser-expansion work that lifts the number doesn't break the test; the test fails only when the gate would now pass (i.e., when this deferral ADR should be superseded by a successful promotion).

adr_0121_no_signed_expert_claim_for_math

docs/reviewers.yaml carries no expert_claims entry for mathematics_logic. (The expert_claims key may not yet exist in the registry — both states are valid representations of "not promoted.")

adr_0121_other_obligations_still_pass

The 10 ADR-0114a obligations all still pass for the gsm8k_math lane. The deferral is on the correct_rate gate alone, not on a substrate regression. Indirectly tested by the existing Phase 5 test suite (74 cases on main); this ADR adds a single roll-up assertion that the existing tests collectively pass.

adr_0121_wrong_zero_holds_against_real_gsm8k

The runner's wrong count on the sealed GSM8K test is 0. This is the load-bearing positive claim of ADR-0121 — even though the contract refuses, the wrong-zero discipline holds. Tested directly.

---

Acceptance evidence

• This ADR file exists in docs/decisions/ and is linked from docs/decisions/README.md
• tests/test_adr_0121_math_expert_deferred.py pins the five invariants above; passes locally with CORE_HOLDOUT_KEY set; skips the decryption-dependent invariants gracefully without it
• ledger_report() continues to report mathematics_logic at status == "audit-passed" (no regression)
• README updated to mention the deferral

---

Consequences

• The expert ledger tier remains unoccupied. No domain is at predicates.expert == True on main.
• ADR-0120's contract has now demonstrated its load-bearing behavior by refusing once. Same pattern as ADR-0107 demonstrated for audit-passed.
• The parser-expansion arc (Phase X of ADR-0114) is now the named bottleneck for the first expert promotion. Each expansion ADR is a concrete piece of work; the path is no longer abstract.
• External readers can read this ADR and see (a) the substrate is complete, (b) the gate refused honestly, (c) the specific number that needs to lift. That's a stronger story than "we promoted the domain because the substrate exists."

---

Out of scope

• The parser-expansion ADRs themselves. Each construction class gets its own scoped ADR (estimated 4–8 total before the gate passes).
• The evaluate_expert_promotion module + reporting integration (ADR-0120a, separate PR).
• Any other domain's promotion (physics, systems_software). Both stay at audit-passed. Their own promotion ADRs are future work after mathematics_logic succeeds.
• Raising the 0.60 floor or tightening ε. Future ADR-0120 amendments if the calibration is ever revisited.
• The second-domain choice (symbolic_logic is recommended per ADR-0120 §"Open candidate directions" Phase 4; that work begins after this deferral resolves).

---

Why "the contract refused" is the right outcome

The strongest claim CORE can make about any future expert promotion is: "this promotion happened because the gate accepted, not because the substrate existed." That distinction requires the gate to have shown it CAN refuse. ADR-0107 demonstrated this for audit-passed; ADR-0121 now demonstrates it for expert.

Every future expert claim — math, symbolic_logic, medical, any domain — inherits this credibility. The reader who looks at a future expert row in the ledger sees an artifact that:

1. Cleared all 10 ADR-0114a obligations
2. Cleared the contract-level gates (including correct_rate ≥ 0.60)
3. Reproduces from a signed digest
4. Could have been refused under the same gate (because the gate refused at least once before)

The deferral here is what makes that fourth property hold. Without it, "expert" would be a status the project hands out by default to any substrate-complete lane. With it, expert is something the gate decided to accept — and the gate has a track record of saying no.

Both the audit-passed and expert tiers now have at least one worked refusal in their history. The promotion machinery is demonstrably load-bearing across both ceilings.