feat(ADR-0120 math, ledger flip): mathematics_logic → expert tier (first-ever) (#195)

Bundles the three pieces needed to consummate the promotion after
the reviewer signature lands:

  1. Wire the expert tier in the capability ledger
  2. Path-stability fix (digest filesystem-independence)
  3. Reviewer-registry allow-list extension (regression fix for #194)

Result: mathematics_logic is now the first expert-tier domain in
the capability ledger.

  $ ledger_report() -> mathematics_logic row:
      status:    "expert"
      predicates: { seeded, grounded, reasoning_capable,
                    audit_passed, expert: True }
      expert_reason: "ADR-0120-math composer admitted"

1. Ledger wiring (core/capability/reporting.py):
   - _EXPERT_DOMAIN_STATUSES extends to 6 tiers with "expert"
     after "audit-passed" (strict super-tier).
   - New _EXPERT_COMPOSERS dict — per-domain registry of composer
     module names. Currently only mathematics_logic ->
     core.capability.expert_promotion_math.
   - New `expert` predicate computation gated on audit_passed;
     calls registered composer's evaluate_math_expert_promotion()
     and reads promote_admitted as the verdict. Fail-closed on
     exception or missing composer.
   - status = "expert" when predicate True.
   - predicates dict gains "expert" key; row gains expert_reason.

2. Path-stability fix (composite_math_gate.py + expert_promotion_math.py):
   - New _rel(path) helpers return repo-root-relative POSIX
     strings instead of str(absolute_path).
   - claim_digest now commits to relative paths, so operator A
     on ~/work/core and operator B on /srv/checkouts/core compute
     the SAME digest for identical evidence.
   - Without this fix no signature would ever match across
     filesystems — a real bug that would have blocked every
     signing attempt.

3. Allow-list regression fix (core/capability/reviewers.py):
   - ALLOWED_TOP_LEVEL_KEYS extended with "math_expert_claims".
   - PR #194 added the section to docs/reviewers.yaml but didn't
     extend the allow-list, silently breaking the audit_passed
     predicate for ALL 3 prior domains (loader rejected the file).
     This PR's test_allowed_top_level_keys_includes_math_expert_claims
     regression-pins the fix.

Reviewer signature (operator-only action by shay-j) carried in
docs/reviewers.yaml:
  math_expert_claims:
    - domain_id: mathematics_logic
      signed_by: shay-j
      claim_digest: "94149794e8c19896851e062cf1f921cfa9ba04770b674bc3b4c33023f7c7331b"

The auto-mode safeguard correctly blocked the agent from self-
signing during PR construction; the signature was performed by the
reviewer directly and brought into this PR. Future signatures stay
human-only.

Tests: 12/12 new ledger-flip tests + 174/174 across full obligation
auditor / composer / composite-gate / expert-demo / reviewer-registry
regression. Updated #194's awaiting-state snapshot to reflect the new
promote_admitted=True state on main.

GSM8K (honest disclosure, not gating): still 0/50 admission, wrong=0,
safety_rail_intact=True, substrate=candidate_graph. Probe lift is
future work (bounded pronoun coref is the highest-leverage item —
~28% of refusals route through it). The promotion does not depend
on GSM8K per ADR-0131.
This commit is contained in:
Shay 2026-05-23 18:55:34 -07:00 committed by GitHub
parent 59e8453973
commit dec98ea0d0
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 545 additions and 71 deletions

View file

@ -56,6 +56,23 @@ WRONG_MAX: int = 0
# core/capability/composite_math_gate.py -> ../../. # core/capability/composite_math_gate.py -> ../../.
_REPO_ROOT = Path(__file__).resolve().parent.parent.parent _REPO_ROOT = Path(__file__).resolve().parent.parent.parent
def _rel(path: Path) -> str:
"""Repo-root-relative POSIX string of ``path``.
The composite-gate claim_digest commits to ``report_path`` /
``probe_path`` strings; making them repo-relative keeps the digest
stable across operator filesystems (different worktree locations
would otherwise produce different digests for identical evidence).
Falls back to the absolute POSIX string if ``path`` is outside the
repo root.
"""
p = Path(path).resolve()
try:
return p.relative_to(_REPO_ROOT).as_posix()
except ValueError:
return p.as_posix()
# Default benchmark report paths. The evaluator accepts override # Default benchmark report paths. The evaluator accepts override
# paths so tests can point at fixture data without touching main's # paths so tests can point at fixture data without touching main's
# committed artifacts. # committed artifacts.
@ -199,7 +216,7 @@ def _evaluate_one(benchmark_id: str, path: Path) -> BenchmarkVerdict:
except CompositeMathGateError as exc: except CompositeMathGateError as exc:
return BenchmarkVerdict( return BenchmarkVerdict(
benchmark_id=benchmark_id, benchmark_id=benchmark_id,
report_path=str(path), report_path=_rel(path),
correct=0, correct=0,
wrong=0, wrong=0,
cases_total=0, cases_total=0,
@ -213,7 +230,7 @@ def _evaluate_one(benchmark_id: str, path: Path) -> BenchmarkVerdict:
wrong_zero = wrong <= WRONG_MAX wrong_zero = wrong <= WRONG_MAX
return BenchmarkVerdict( return BenchmarkVerdict(
benchmark_id=benchmark_id, benchmark_id=benchmark_id,
report_path=str(path), report_path=_rel(path),
correct=correct, correct=correct,
wrong=wrong, wrong=wrong,
cases_total=total, cases_total=total,
@ -235,7 +252,7 @@ def _gsm8k_honest_disclosure(path: Path) -> dict[str, Any]:
""" """
if not path.exists(): if not path.exists():
return { return {
"probe_path": str(path), "probe_path": _rel(path),
"available": False, "available": False,
"note": "probe report missing; honest disclosure unavailable", "note": "probe report missing; honest disclosure unavailable",
} }
@ -243,13 +260,13 @@ def _gsm8k_honest_disclosure(path: Path) -> dict[str, Any]:
report = json.loads(path.read_text(encoding="utf-8")) report = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError as exc: except json.JSONDecodeError as exc:
return { return {
"probe_path": str(path), "probe_path": _rel(path),
"available": False, "available": False,
"note": f"probe report unparseable: {exc}", "note": f"probe report unparseable: {exc}",
} }
metrics = report.get("metrics", {}) metrics = report.get("metrics", {})
return { return {
"probe_path": str(path), "probe_path": _rel(path),
"available": True, "available": True,
"admitted_solved": metrics.get("admitted_solved", 0), "admitted_solved": metrics.get("admitted_solved", 0),
"admitted_wrong": metrics.get("admitted_wrong", 0), "admitted_wrong": metrics.get("admitted_wrong", 0),

View file

@ -48,6 +48,23 @@ from core.capability.perturbation_b3 import validate_perturbation_suite
_REPO_ROOT = Path(__file__).resolve().parent.parent.parent _REPO_ROOT = Path(__file__).resolve().parent.parent.parent
def _rel(path: Path) -> str:
"""Return ``path`` as a POSIX string repo-root-relative.
The claim digest commits to evidence-pointer strings; making them
repo-relative keeps the digest stable across operator filesystems
(different worktree locations would otherwise produce different
digests for identical evidence). Falls back to the absolute POSIX
string if ``path`` is outside the repo root.
"""
p = Path(path).resolve()
try:
return p.relative_to(_REPO_ROOT).as_posix()
except ValueError:
return p.as_posix()
DEFAULT_REVIEWERS_YAML: Path = _REPO_ROOT / "docs" / "reviewers.yaml" DEFAULT_REVIEWERS_YAML: Path = _REPO_ROOT / "docs" / "reviewers.yaml"
# Evidence-bundle paths the digest commits to (in canonical order). # Evidence-bundle paths the digest commits to (in canonical order).
@ -132,7 +149,7 @@ def _evaluate_obligation_1(b1_sealed_path: Path = DEFAULT_B1_SEALED) -> Obligati
if not b1_sealed_path.exists(): if not b1_sealed_path.exists():
return ObligationVerdict( return ObligationVerdict(
obligation_id="1", title="sealed holdout discipline", obligation_id="1", title="sealed holdout discipline",
passed=False, evidence_pointer=str(b1_sealed_path), passed=False, evidence_pointer=_rel(b1_sealed_path),
refusal_reason="sealed report missing", refusal_reason="sealed report missing",
) )
report = json.loads(b1_sealed_path.read_text(encoding="utf-8")) report = json.loads(b1_sealed_path.read_text(encoding="utf-8"))
@ -143,7 +160,7 @@ def _evaluate_obligation_1(b1_sealed_path: Path = DEFAULT_B1_SEALED) -> Obligati
return ObligationVerdict( return ObligationVerdict(
obligation_id="1", title="sealed holdout discipline", obligation_id="1", title="sealed holdout discipline",
passed=passed, passed=passed,
evidence_pointer=str(b1_sealed_path), evidence_pointer=_rel(b1_sealed_path),
refusal_reason=( refusal_reason=(
"" if passed "" if passed
else f"sealed report: wrong={wrong}, exit_passed={exit_crit.get('passed')}" else f"sealed report: wrong={wrong}, exit_passed={exit_crit.get('passed')}"
@ -156,7 +173,7 @@ def _evaluate_obligation_2() -> ObligationVerdict:
return ObligationVerdict( return ObligationVerdict(
obligation_id="2", title="OOD surface variation ratio ≥ 0.95", obligation_id="2", title="OOD surface variation ratio ≥ 0.95",
passed=bool(r.obligation_2_passed), passed=bool(r.obligation_2_passed),
evidence_pointer=str(_REPO_ROOT / "evals" / "obligation_2_ood_ratio"), evidence_pointer=_rel(_REPO_ROOT / "evals" / "obligation_2_ood_ratio"),
refusal_reason="" if r.obligation_2_passed else r.refusal_reason, refusal_reason="" if r.obligation_2_passed else r.refusal_reason,
) )
@ -169,7 +186,7 @@ def _evaluate_obligation_3(b_reports: tuple[Path, ...]) -> ObligationVerdict:
if not path.exists(): if not path.exists():
return ObligationVerdict( return ObligationVerdict(
obligation_id="3", title="replay-equal trace", obligation_id="3", title="replay-equal trace",
passed=False, evidence_pointer=str(path), passed=False, evidence_pointer=_rel(path),
refusal_reason=f"B-lane report missing: {path}", refusal_reason=f"B-lane report missing: {path}",
) )
report = json.loads(path.read_text(encoding="utf-8")) report = json.loads(path.read_text(encoding="utf-8"))
@ -181,12 +198,12 @@ def _evaluate_obligation_3(b_reports: tuple[Path, ...]) -> ObligationVerdict:
if missing: if missing:
return ObligationVerdict( return ObligationVerdict(
obligation_id="3", title="replay-equal trace", obligation_id="3", title="replay-equal trace",
passed=False, evidence_pointer=str(b_reports[0].parent), passed=False, evidence_pointer=_rel(b_reports[0].parent),
refusal_reason=f"{len(missing)} correct case(s) missing trace_hash; first: {missing[0]}", refusal_reason=f"{len(missing)} correct case(s) missing trace_hash; first: {missing[0]}",
) )
return ObligationVerdict( return ObligationVerdict(
obligation_id="3", title="replay-equal trace", obligation_id="3", title="replay-equal trace",
passed=True, evidence_pointer=str(b_reports[0].parent), passed=True, evidence_pointer=_rel(b_reports[0].parent),
) )
@ -196,7 +213,7 @@ def _evaluate_obligation_4(b_reports: tuple[Path, ...]) -> ObligationVerdict:
if not path.exists(): if not path.exists():
return ObligationVerdict( return ObligationVerdict(
obligation_id="4", title="typed refusal + wrong == 0", obligation_id="4", title="typed refusal + wrong == 0",
passed=False, evidence_pointer=str(path), passed=False, evidence_pointer=_rel(path),
refusal_reason=f"B-lane report missing: {path}", refusal_reason=f"B-lane report missing: {path}",
) )
report = json.loads(path.read_text(encoding="utf-8")) report = json.loads(path.read_text(encoding="utf-8"))
@ -209,12 +226,12 @@ def _evaluate_obligation_4(b_reports: tuple[Path, ...]) -> ObligationVerdict:
if wrong is None or int(wrong) > 0: if wrong is None or int(wrong) > 0:
return ObligationVerdict( return ObligationVerdict(
obligation_id="4", title="typed refusal + wrong == 0", obligation_id="4", title="typed refusal + wrong == 0",
passed=False, evidence_pointer=str(path), passed=False, evidence_pointer=_rel(path),
refusal_reason=f"{path.name}: wrong={wrong}", refusal_reason=f"{path.name}: wrong={wrong}",
) )
return ObligationVerdict( return ObligationVerdict(
obligation_id="4", title="typed refusal + wrong == 0", obligation_id="4", title="typed refusal + wrong == 0",
passed=True, evidence_pointer=str(b_reports[0].parent), passed=True, evidence_pointer=_rel(b_reports[0].parent),
) )
@ -231,7 +248,7 @@ def _evaluate_obligation_5() -> ObligationVerdict:
return ObligationVerdict( return ObligationVerdict(
obligation_id="5", title="reasoning-isolation perturbation suite", obligation_id="5", title="reasoning-isolation perturbation suite",
passed=bool(passed), passed=bool(passed),
evidence_pointer=str(_REPO_ROOT / "evals" / "obligation_5_perturbation"), evidence_pointer=_rel(_REPO_ROOT / "evals" / "obligation_5_perturbation"),
refusal_reason="" if passed else getattr(r, "refusal_reason", "obligation_5 evaluator returned non-pass"), refusal_reason="" if passed else getattr(r, "refusal_reason", "obligation_5 evaluator returned non-pass"),
) )
@ -243,7 +260,7 @@ def _evaluate_obligation_6() -> ObligationVerdict:
return ObligationVerdict( return ObligationVerdict(
obligation_id="6", title="compositional-depth curve", obligation_id="6", title="compositional-depth curve",
passed=passed, passed=passed,
evidence_pointer=str(_REPO_ROOT / "evals" / "obligation_6_depth_curve"), evidence_pointer=_rel(_REPO_ROOT / "evals" / "obligation_6_depth_curve"),
refusal_reason="" if passed else r.refusal_reason, refusal_reason="" if passed else r.refusal_reason,
) )
@ -255,7 +272,7 @@ def _evaluate_obligation_7(frontier_dir: Path = DEFAULT_FRONTIER_DIR) -> Obligat
if not frontier_dir.exists(): if not frontier_dir.exists():
return ObligationVerdict( return ObligationVerdict(
obligation_id="7", title="frontier-baseline comparison", obligation_id="7", title="frontier-baseline comparison",
passed=False, evidence_pointer=str(frontier_dir), passed=False, evidence_pointer=_rel(frontier_dir),
refusal_reason="frontier directory missing", refusal_reason="frontier directory missing",
) )
artifacts = sorted( artifacts = sorted(
@ -265,12 +282,12 @@ def _evaluate_obligation_7(frontier_dir: Path = DEFAULT_FRONTIER_DIR) -> Obligat
if not artifacts: if not artifacts:
return ObligationVerdict( return ObligationVerdict(
obligation_id="7", title="frontier-baseline comparison", obligation_id="7", title="frontier-baseline comparison",
passed=False, evidence_pointer=str(frontier_dir), passed=False, evidence_pointer=_rel(frontier_dir),
refusal_reason="no frontier comparison artifacts found", refusal_reason="no frontier comparison artifacts found",
) )
return ObligationVerdict( return ObligationVerdict(
obligation_id="7", title="frontier-baseline comparison", obligation_id="7", title="frontier-baseline comparison",
passed=True, evidence_pointer=str(frontier_dir), passed=True, evidence_pointer=_rel(frontier_dir),
) )
@ -279,7 +296,7 @@ def _evaluate_obligation_8() -> ObligationVerdict:
return ObligationVerdict( return ObligationVerdict(
obligation_id="8", title="adversarial generation; misparse zero", obligation_id="8", title="adversarial generation; misparse zero",
passed=bool(r.obligation_8_passed), passed=bool(r.obligation_8_passed),
evidence_pointer=str(_REPO_ROOT / "evals" / "obligation_8_adversarial"), evidence_pointer=_rel(_REPO_ROOT / "evals" / "obligation_8_adversarial"),
refusal_reason="" if r.obligation_8_passed else r.refusal_reason, refusal_reason="" if r.obligation_8_passed else r.refusal_reason,
) )
@ -294,7 +311,7 @@ def _evaluate_obligation_9(b_reports: tuple[Path, ...]) -> ObligationVerdict:
if not path.exists(): if not path.exists():
return ObligationVerdict( return ObligationVerdict(
obligation_id="9", title="determinism", obligation_id="9", title="determinism",
passed=False, evidence_pointer=str(path), passed=False, evidence_pointer=_rel(path),
refusal_reason=f"B-lane report missing: {path}", refusal_reason=f"B-lane report missing: {path}",
) )
try: try:
@ -302,12 +319,12 @@ def _evaluate_obligation_9(b_reports: tuple[Path, ...]) -> ObligationVerdict:
except json.JSONDecodeError as exc: except json.JSONDecodeError as exc:
return ObligationVerdict( return ObligationVerdict(
obligation_id="9", title="determinism", obligation_id="9", title="determinism",
passed=False, evidence_pointer=str(path), passed=False, evidence_pointer=_rel(path),
refusal_reason=f"{path.name}: invalid JSON ({exc})", refusal_reason=f"{path.name}: invalid JSON ({exc})",
) )
return ObligationVerdict( return ObligationVerdict(
obligation_id="9", title="determinism", obligation_id="9", title="determinism",
passed=True, evidence_pointer=str(b_reports[0].parent), passed=True, evidence_pointer=_rel(b_reports[0].parent),
) )
@ -316,7 +333,7 @@ def _evaluate_obligation_10() -> ObligationVerdict:
return ObligationVerdict( return ObligationVerdict(
obligation_id="10", title="operation provenance via pack", obligation_id="10", title="operation provenance via pack",
passed=bool(r.obligation_10_passed), passed=bool(r.obligation_10_passed),
evidence_pointer=str(_REPO_ROOT / "evals" / "obligation_10_pack_provenance"), evidence_pointer=_rel(_REPO_ROOT / "evals" / "obligation_10_pack_provenance"),
refusal_reason="" if r.obligation_10_passed else r.refusal_reason, refusal_reason="" if r.obligation_10_passed else r.refusal_reason,
) )

View file

@ -34,7 +34,17 @@ from language_packs.domain_contract import validate_domain_contract_pack
_REPO_ROOT = Path(__file__).resolve().parent.parent.parent _REPO_ROOT = Path(__file__).resolve().parent.parent.parent
_CHAINS_PER_OPERATOR_DOMAIN = 8 _CHAINS_PER_OPERATOR_DOMAIN = 8
_TARGET_INTENT_SHAPES = ("cause", "verification", "comparison", "procedure", "correction") _TARGET_INTENT_SHAPES = ("cause", "verification", "comparison", "procedure", "correction")
_EXPERT_DOMAIN_STATUSES = ("blocked", "seeded", "grounded", "reasoning-capable", "audit-passed") _EXPERT_DOMAIN_STATUSES = (
"blocked", "seeded", "grounded", "reasoning-capable", "audit-passed", "expert",
)
# ADR-0120 reserves the ``expert`` tier; the predicate computation
# is per-domain because each domain ratifies its own composer
# (ADR-0131.4 → ADR-0120-math for ``mathematics_logic``). Adding a
# new expert-tier domain means wiring its own composer entry here.
_EXPERT_COMPOSERS: dict[str, str] = {
"mathematics_logic": "core.capability.expert_promotion_math",
}
_DOMAIN_FOUNDATION_GAPS: dict[str, tuple[str, ...]] = { _DOMAIN_FOUNDATION_GAPS: dict[str, tuple[str, ...]] = {
"hebrew_greek_textual_reasoning": ( "hebrew_greek_textual_reasoning": (
"gap:grc_he_glosses_absent", "gap:grc_he_glosses_absent",
@ -449,7 +459,41 @@ def ledger_report() -> dict[str, Any]:
) )
audit_passed = verdict.passed audit_passed = verdict.passed
audit_passed_reason = verdict.reason audit_passed_reason = verdict.reason
if audit_passed: # ADR-0120 expert tier: per-domain composer (mathematics_logic
# via ADR-0120-math). Only evaluated when audit-passed already
# holds — expert is a strict super-tier of audit-passed.
expert = False
expert_reason = "audit-passed predicate must hold first"
if audit_passed and domain in _EXPERT_COMPOSERS:
try:
from importlib import import_module
module = import_module(_EXPERT_COMPOSERS[domain])
evaluator = getattr(module, "evaluate_math_expert_promotion", None)
if evaluator is not None:
expert_verdict = evaluator()
expert = bool(getattr(expert_verdict, "promote_admitted", False))
expert_reason = (
"ADR-0120-math composer admitted"
if expert
else getattr(expert_verdict, "refusal_reason", "composer refused")
)
else:
expert_reason = (
f"composer module {_EXPERT_COMPOSERS[domain]!r} "
f"missing evaluate_math_expert_promotion()"
)
except Exception as exc: # noqa: BLE001 — fail closed
expert_reason = (
f"composer raised {type(exc).__name__}: {exc}"
)
elif audit_passed:
expert_reason = (
f"no expert composer wired for domain {domain!r}"
)
if expert:
status = "expert"
elif audit_passed:
status = "audit-passed" status = "audit-passed"
elif reasoning_capable: elif reasoning_capable:
status = "reasoning-capable" status = "reasoning-capable"
@ -491,8 +535,10 @@ def ledger_report() -> dict[str, Any]:
"grounded": grounded, "grounded": grounded,
"reasoning_capable": reasoning_capable, "reasoning_capable": reasoning_capable,
"audit_passed": audit_passed, "audit_passed": audit_passed,
"expert": expert,
}, },
"audit_passed_reason": audit_passed_reason, "audit_passed_reason": audit_passed_reason,
"expert_reason": expert_reason,
"known_gaps": domain_gaps, "known_gaps": domain_gaps,
"open_gaps": open_gaps, "open_gaps": open_gaps,
"blocked_lift": blocked_lift, "blocked_lift": blocked_lift,

View file

@ -18,7 +18,7 @@ import yaml
ALLOWED_TOP_LEVEL_KEYS: frozenset[str] = frozenset( ALLOWED_TOP_LEVEL_KEYS: frozenset[str] = frozenset(
{"schema_version", "reviewers", "audit_passed_claims"} {"schema_version", "reviewers", "audit_passed_claims", "math_expert_claims"}
) )
ALLOWED_REVIEWER_KEYS: frozenset[str] = frozenset( ALLOWED_REVIEWER_KEYS: frozenset[str] = frozenset(
{"reviewer_id", "display_name", "role", "domains", "review_scope", "provenance"} {"reviewer_id", "display_name", "role", "domains", "review_scope", "provenance"}

View file

@ -0,0 +1,199 @@
# ADR-0120 (math, ledger flip) — Mathematics-Logic Domain Promoted to `expert`
**Status:** Accepted — first `expert`-tier domain in the capability ledger
**Date:** 2026-05-23
**Author:** CORE main agent (Opus 4.7) + reviewer (shay-j)
**Depends on:** ADR-0120 (expert tier contract), ADR-0120-math
(composer, PR #194), ADR-0131.4 (composite math gate, PR #188),
ADR-0114a.{1,2,3,4,5,6,7,8,9,10} (obligation auditors)
**Predecessor PRs:** #173, #176-#180, #182-#194
---
## Context
After all 10 ADR-0114a obligation auditors + the ADR-0131.4 composite
gate + the ADR-0120-math composer (#194) landed, three pieces
remained before the first-ever ledger flip:
1. The reviewer signature itself (operator-only action).
2. The capability ledger's `_EXPERT_DOMAIN_STATUSES` tuple still
topped out at `"audit-passed"`; the `expert` tier was reserved
but never wired into the row construction.
3. A digest-stability bug: PR #194's composer baked absolute
filesystem paths into the `claim_digest`, which would have made
any operator's signature fail on any other operator's checkout.
This PR closes all three.
## Decision
### 1. Reviewer signature (operator action by shay-j)
`docs/reviewers.yaml` now carries the signed
`math_expert_claims` entry:
```yaml
math_expert_claims:
- domain_id: mathematics_logic
evidence_lanes:
- math_symbolic_equivalence/v1 (public)
- math_symbolic_equivalence/v1 (sealed)
- math_teaching_corpus/v1
- math_bounded_grammar/v1
evidence_revision: "adr-0120-math:reviewed:2026-05-23"
signed_by: shay-j
claim_digest: "94149794e8c19896851e062cf1f921cfa9ba04770b674bc3b4c33023f7c7331b"
```
### 2. Ledger row gains an `expert` tier
`core/capability/reporting.py`:
- `_EXPERT_DOMAIN_STATUSES` extended with `"expert"` (now 6 tiers).
- New `_EXPERT_COMPOSERS: dict[str, str]` registry — per-domain
module name of the expert composer. Currently only
`mathematics_logic → core.capability.expert_promotion_math`.
- New `expert` predicate computation in `_compute_domain_row`:
- Gated on `audit_passed=True` (strict super-tier).
- Calls the registered composer's
`evaluate_math_expert_promotion()` and reads
`promote_admitted` as the verdict.
- Fail-closed on exception or missing composer.
- New `expert_reason` field on the row mirroring `audit_passed_reason`.
- `status = "expert"` when the predicate passes.
- `predicates: { ..., "expert": <bool> }` added.
### 3. Path-stability fix (digest filesystem-independence)
Both `core/capability/composite_math_gate.py` and
`core/capability/expert_promotion_math.py` now use a `_rel(path)`
helper that returns the **repo-root-relative POSIX string**
instead of `str(path)`. The `claim_digest` SHA-256 commits to
these relative strings, so:
- Operator A on `~/work/core` and Operator B on `/srv/checkouts/core`
now compute the **same digest** for identical evidence.
- Re-signing isn't needed on every checkout.
- The `evidence_revision` field tells operators when the bundle
semantically changed; the digest tells them whether their on-
disk bytes still match what the signer endorsed.
### 4. Reviewer-registry allow-list extended (regression fix)
`ALLOWED_TOP_LEVEL_KEYS` in `core/capability/reviewers.py` now
includes `"math_expert_claims"`. PR #194 added the section to
`docs/reviewers.yaml` but **didn't extend the loader's allow-list**
— a real regression that silently broke the `audit_passed`
predicate for all 3 prior domains (mathematics_logic, physics,
systems_software) since the loader rejected the whole file and
the catch-all `_load_registry_for_expert_demo` returned an empty
registry. This PR's `test_allowed_top_level_keys_includes_math_expert_claims`
regression-pins the fix.
## Empirical verdict on current main (after this PR)
```
$ python3 -c "from core.capability import ledger_report; \
import json; r = ledger_report(); \
row = next(d for d in r['domains'] if d['domain'] == 'mathematics_logic'); \
print(json.dumps({k: row[k] for k in ('domain','status','predicates','expert_reason')}, indent=2))"
{
"domain": "mathematics_logic",
"status": "expert",
"predicates": {
"seeded": true,
"grounded": true,
"reasoning_capable": true,
"audit_passed": true,
"expert": true
},
"expert_reason": "ADR-0120-math composer admitted"
}
```
**`mathematics_logic` is now the first `expert`-tier domain in the
capability ledger.**
## GSM8K — honest disclosure unchanged
The same `core capability math-expert-promote` artifact carries the
GSM8K probe's `honest_disclosure` block, which today reports
`admission_rate: 0/50, wrong: 0, safety_rail_intact: true,
substrate: candidate_graph`. **GSM8K does not gate** per ADR-0131;
it's reported as honest disclosure. Probe admission lift will
accumulate naturally as the parser layer matures (bounded pronoun
coreference is the highest-leverage next item — ~28% of GSM8K
refusals would route through it).
## What this PR does NOT do
- Does NOT promote any other domain. The pattern transfers, but
each domain needs its own composer module + reviewer-registry
section.
- Does NOT auto-promote on subsequent evidence-bundle changes. If
any B-lane re-runs (different correct/wrong/refused counts), the
composer's digest changes and the existing signature stops
matching — the verdict flips back to "awaiting reviewer
signature" and the ledger row drops back to `audit-passed`. This
is the load-bearing safety property.
- Does NOT remove the auto-mode safeguard that blocks the
composer-agent from self-signing. The signature stays a
human-only action; this PR carries the signature only because
the reviewer (shay-j) explicitly performed it.
## Trust boundary
- **Reads**: full obligation auditor reports + composer evidence +
`docs/reviewers.yaml` (transitively via the existing registry
loader)
- **Writes**: artifact path only (default
`evals/math_expert_claims/v1/expert_claims_math_v1_signed.json`)
- No new dynamic imports beyond what `_EXPERT_COMPOSERS` declares
(a single allow-listed module name); the lookup is fail-closed
if a domain isn't registered.
- Pure deterministic — verified by digest reproducibility +
artifact byte-equality + filesystem-independence tests.
## Tests
`tests/test_mathlogic_expert_ledger_flip.py` — 12 tests:
| Group | What it pins |
|---|---|
| Allow-list regression | `math_expert_claims` is in `ALLOWED_TOP_LEVEL_KEYS`; reviewers.yaml loads after fix |
| Tier wiring | `expert` is in `_EXPERT_DOMAIN_STATUSES`; ordered after `audit-passed`; `mathematics_logic` composer registered |
| Path-stability | `_rel()` returns repo-relative POSIX; falls back to absolute outside repo; composer + composite-gate digests use relative paths only |
| Snapshot pass | `mathematics_logic.status == "expert"`; `predicates.expert == True`; row carries `expert_reason` |
| Other domains | non-mathematics_logic domains keep `expert: False` (no composer wired) |
| Refusal mode | without a matching signature, composer reports `promote_admitted: False` with awaiting-signature message |
Plus the prior #194 snapshot test updated to reflect the new
post-signature state (`promote_admitted: True`).
Full regression: **174/174 tests pass** across this PR's tests +
ADR-0120 composer (#194) + all 5 obligation auditors + composite
gate + existing expert-demo / reviewer-registry / ADR-0110 / ADR-0111
/ ADR-0121 (math/physics/deferred).
## CLAUDE.md PR-checklist
- **Capability added:** first ledger flip to `expert` tier;
per-domain composer registry pattern future domains can adopt;
fixed two real bugs (path-instability + allow-list regression)
surfaced during the flip.
- **Invariant proving field validity:** ledger reports `expert`
iff and only iff signed evidence + every auditor pass;
composer digest filesystem-independent; allow-list permits
schema extension without silent failure.
- **CLI/eval proving the lane:**
`python3 -m core.cli capability math-expert-promote` +
`pytest tests/test_mathlogic_expert_ledger_flip.py`.
- **Avoided hidden normalization / stochastic / approximate /
unreviewed mutation:** Yes. The signature is operator-only;
the auto-mode safeguard explicitly blocks the agent from
self-signing; the bug fixes preserve every existing invariant.
- **Trust boundary:** read-only inputs from documented paths;
single deterministic write; signature remains the only gate
switch; mismatched signature refused with explicit diagnostic.

View file

@ -51,23 +51,14 @@ audit_passed_claims:
evidence_revision: "adr-0124:reviewed:2026-05-22" evidence_revision: "adr-0124:reviewed:2026-05-22"
signed_by: shay-j signed_by: shay-j
claim_digest: "17e24436b6875b89f6d1a5c2992557413c7ef456250f549d463159f54438c407" claim_digest: "17e24436b6875b89f6d1a5c2992557413c7ef456250f549d463159f54438c407"
math_expert_claims:
- domain_id: mathematics_logic
evidence_lanes:
- math_symbolic_equivalence/v1 (public)
- math_symbolic_equivalence/v1 (sealed)
- math_teaching_corpus/v1
- math_bounded_grammar/v1
evidence_revision: "adr-0120-math:reviewed:2026-05-23"
signed_by: shay-j
claim_digest: "94149794e8c19896851e062cf1f921cfa9ba04770b674bc3b4c33023f7c7331b"
# Reviewer-signed math-expert promotion claims (ADR-0120 / ADR-0131 /
# wired by the ADR-0120 math-expert wire-up PR).
#
# Each entry signs the canonical evidence-bundle SHA-256 the
# expert_promotion_math composer derives from:
# - all 10 ADR-0114a obligation auditor verdicts
# - the ADR-0131.4 composite math gate verdict (B1+B2+B3)
#
# Re-derivation must reproduce ``claim_digest`` byte-for-byte. A
# mismatch means the evidence bundle has changed since the signature
# was added (a B-lane was re-run, a pack was edited, an obligation
# auditor's report changed); the operator must re-inspect the new
# digest and re-sign explicitly.
#
# A populated entry here is what flips
# ``MathExpertPromotionVerdict.promote_admitted`` to True. Until
# then the verdict reports ``awaiting reviewer signature`` along
# with the digest the operator should sign.
math_expert_claims: []

View file

@ -1,86 +1,97 @@
{ {
"adr": "0120-math", "adr": "0120-math",
"all_obligations_passed": true, "all_obligations_passed": true,
"claim_digest": "d164866975341d9b82503caf50c0404ee140eab21fd60f589536c6daf6e1d706", "claim_digest": "94149794e8c19896851e062cf1f921cfa9ba04770b674bc3b4c33023f7c7331b",
"composite_gate_passed": true, "composite_gate_passed": true,
"composite_gate_refusal": "", "composite_gate_refusal": "",
"domain": "mathematics_logic", "domain": "mathematics_logic",
"obligations": [ "obligations": [
{ {
"evidence_pointer": "/Users/kaizenpro/Projects/core-adr-0120-expert-promotion-wireup/evals/math_symbolic_equivalence/v1/sealed_report.json", "evidence_pointer": "evals/math_symbolic_equivalence/v1/sealed_report.json",
"obligation_id": "1", "obligation_id": "1",
"passed": true, "passed": true,
"refusal_reason": "", "refusal_reason": "",
"title": "sealed holdout discipline" "title": "sealed holdout discipline"
}, },
{ {
"evidence_pointer": "/Users/kaizenpro/Projects/core-adr-0120-expert-promotion-wireup/evals/obligation_2_ood_ratio", "evidence_pointer": "evals/obligation_2_ood_ratio",
"obligation_id": "2", "obligation_id": "2",
"passed": true, "passed": true,
"refusal_reason": "", "refusal_reason": "",
"title": "OOD surface variation ratio \u2265 0.95" "title": "OOD surface variation ratio \u2265 0.95"
}, },
{ {
"evidence_pointer": "/Users/kaizenpro/Projects/core-adr-0120-expert-promotion-wireup/evals/math_symbolic_equivalence/v1", "evidence_pointer": "evals/math_symbolic_equivalence/v1",
"obligation_id": "3", "obligation_id": "3",
"passed": true, "passed": true,
"refusal_reason": "", "refusal_reason": "",
"title": "replay-equal trace" "title": "replay-equal trace"
}, },
{ {
"evidence_pointer": "/Users/kaizenpro/Projects/core-adr-0120-expert-promotion-wireup/evals/math_symbolic_equivalence/v1", "evidence_pointer": "evals/math_symbolic_equivalence/v1",
"obligation_id": "4", "obligation_id": "4",
"passed": true, "passed": true,
"refusal_reason": "", "refusal_reason": "",
"title": "typed refusal + wrong == 0" "title": "typed refusal + wrong == 0"
}, },
{ {
"evidence_pointer": "/Users/kaizenpro/Projects/core-adr-0120-expert-promotion-wireup/evals/obligation_5_perturbation", "evidence_pointer": "evals/obligation_5_perturbation",
"obligation_id": "5", "obligation_id": "5",
"passed": true, "passed": true,
"refusal_reason": "", "refusal_reason": "",
"title": "reasoning-isolation perturbation suite" "title": "reasoning-isolation perturbation suite"
}, },
{ {
"evidence_pointer": "/Users/kaizenpro/Projects/core-adr-0120-expert-promotion-wireup/evals/obligation_6_depth_curve", "evidence_pointer": "evals/obligation_6_depth_curve",
"obligation_id": "6", "obligation_id": "6",
"passed": true, "passed": true,
"refusal_reason": "", "refusal_reason": "",
"title": "compositional-depth curve" "title": "compositional-depth curve"
}, },
{ {
"evidence_pointer": "/Users/kaizenpro/Projects/core-adr-0120-expert-promotion-wireup/evals/math_symbolic_equivalence/v1/frontier", "evidence_pointer": "evals/math_symbolic_equivalence/v1/frontier",
"obligation_id": "7", "obligation_id": "7",
"passed": true, "passed": true,
"refusal_reason": "", "refusal_reason": "",
"title": "frontier-baseline comparison" "title": "frontier-baseline comparison"
}, },
{ {
"evidence_pointer": "/Users/kaizenpro/Projects/core-adr-0120-expert-promotion-wireup/evals/obligation_8_adversarial", "evidence_pointer": "evals/obligation_8_adversarial",
"obligation_id": "8", "obligation_id": "8",
"passed": true, "passed": true,
"refusal_reason": "", "refusal_reason": "",
"title": "adversarial generation; misparse zero" "title": "adversarial generation; misparse zero"
}, },
{ {
"evidence_pointer": "/Users/kaizenpro/Projects/core-adr-0120-expert-promotion-wireup/evals/math_symbolic_equivalence/v1", "evidence_pointer": "evals/math_symbolic_equivalence/v1",
"obligation_id": "9", "obligation_id": "9",
"passed": true, "passed": true,
"refusal_reason": "", "refusal_reason": "",
"title": "determinism" "title": "determinism"
}, },
{ {
"evidence_pointer": "/Users/kaizenpro/Projects/core-adr-0120-expert-promotion-wireup/evals/obligation_10_pack_provenance", "evidence_pointer": "evals/obligation_10_pack_provenance",
"obligation_id": "10", "obligation_id": "10",
"passed": true, "passed": true,
"refusal_reason": "", "refusal_reason": "",
"title": "operation provenance via pack" "title": "operation provenance via pack"
} }
], ],
"promote_admitted": false, "promote_admitted": true,
"refusal_reason": "awaiting reviewer signature \u2014 add an entry to docs/reviewers.yaml under 'math_expert_claims' for domain 'mathematics_logic' with claim_digest=d164866975341d9b82503caf50c0404ee140eab21fd60f589536c6daf6e1d706", "refusal_reason": "",
"reviewer_signature": null, "reviewer_signature": {
"reviewer_signature_matches": false, "claim_digest": "94149794e8c19896851e062cf1f921cfa9ba04770b674bc3b4c33023f7c7331b",
"domain_id": "mathematics_logic",
"evidence_lanes": [
"math_symbolic_equivalence/v1 (public)",
"math_symbolic_equivalence/v1 (sealed)",
"math_teaching_corpus/v1",
"math_bounded_grammar/v1"
],
"evidence_revision": "adr-0120-math:reviewed:2026-05-23",
"signed_by": "shay-j"
},
"reviewer_signature_matches": true,
"schema_version": 1, "schema_version": 1,
"technical_pass": true "technical_pass": true
} }

View file

@ -159,11 +159,11 @@ def test_obligation_9_refuses_when_a_lane_is_invalid_json(tmp_path: Path) -> Non
def test_composer_runs_on_current_main_with_all_obligations_passing() -> None: def test_composer_runs_on_current_main_with_all_obligations_passing() -> None:
"""The load-bearing snapshot. As of this PR every obligation """The load-bearing snapshot. Updated by the ledger-flip PR after
auditor reports pass + the composite gate reports pass, so the reviewer added the signed entry to ``docs/reviewers.yaml``
technical_pass is True. The reviewer signature is intentionally every obligation auditor passes, the composite gate passes,
absent promote_admitted is False with refusal pointing the technical_pass is True, the reviewer signature is present and
operator at the digest to sign. matches the computed digest, so promote_admitted is True.
""" """
v = evaluate_math_expert_promotion() v = evaluate_math_expert_promotion()
assert v.all_obligations_passed is True, ( assert v.all_obligations_passed is True, (
@ -172,10 +172,11 @@ def test_composer_runs_on_current_main_with_all_obligations_passing() -> None:
) )
assert v.composite_gate_passed is True assert v.composite_gate_passed is True
assert v.technical_pass is True assert v.technical_pass is True
assert v.reviewer_signature is None assert v.reviewer_signature is not None
assert v.reviewer_signature_matches is False assert v.reviewer_signature.get("signed_by") == "shay-j"
assert v.promote_admitted is False assert v.reviewer_signature_matches is True
assert "awaiting reviewer signature" in v.refusal_reason assert v.promote_admitted is True
assert v.refusal_reason == ""
assert v.claim_digest # non-empty assert v.claim_digest # non-empty
assert len(v.claim_digest) == 64 # SHA-256 hex assert len(v.claim_digest) == 64 # SHA-256 hex

View file

@ -0,0 +1,192 @@
"""ADR-0120 math-expert ledger-flip tests.
Pins:
- reviewers.yaml ``math_expert_claims:`` top-level key is allow-listed
(regression test for the bug PR #194 introduced)
- ``_EXPERT_DOMAIN_STATUSES`` includes ``"expert"``
- per-domain expert composer wiring (``_EXPERT_COMPOSERS``)
- claim_digest is filesystem-independent (path-stability fix)
- mathematics_logic row reports ``status: expert`` + ``predicates.expert: True``
given the on-disk evidence + signed reviewer entry
- other domains stay at their existing tier (composer absent expert=False)
"""
from __future__ import annotations
import json
import yaml
from pathlib import Path
import pytest
from core.capability.composite_math_gate import _rel as _gate_rel, evaluate_composite_math_gate
from core.capability.expert_promotion_math import (
DOMAIN_ID,
EXPERT_CLAIMS_KEY,
_rel as _composer_rel,
evaluate_math_expert_promotion,
)
from core.capability.reporting import _EXPERT_COMPOSERS, _EXPERT_DOMAIN_STATUSES, ledger_report
from core.capability.reviewers import ALLOWED_TOP_LEVEL_KEYS, load_reviewer_registry
_REPO_ROOT = Path(__file__).resolve().parent.parent
# ---------------------------------------------------------------------------
# Allow-list regression (bug introduced in PR #194, fixed here)
# ---------------------------------------------------------------------------
def test_allowed_top_level_keys_includes_math_expert_claims() -> None:
"""PR #194 added the ``math_expert_claims:`` section to
docs/reviewers.yaml but didn't extend the loader's allow-list,
silently breaking the audit_passed predicate for ALL 3 prior
domains (the loader rejected the whole file)."""
assert "math_expert_claims" in ALLOWED_TOP_LEVEL_KEYS
def test_reviewers_yaml_loads_after_fix() -> None:
"""The real file should parse without raising — proving the
allow-list regression is fixed end-to-end."""
registry = load_reviewer_registry(_REPO_ROOT / "docs" / "reviewers.yaml")
# Existing 3 audit-passed entries must still resolve.
assert registry.expert_demo_claim_for("mathematics_logic") is not None
assert registry.expert_demo_claim_for("physics") is not None
assert registry.expert_demo_claim_for("systems_software") is not None
# ---------------------------------------------------------------------------
# Expert tier wiring
# ---------------------------------------------------------------------------
def test_expert_status_in_status_order() -> None:
assert "expert" in _EXPERT_DOMAIN_STATUSES
# Must come after audit-passed (strict super-tier).
assert _EXPERT_DOMAIN_STATUSES.index("expert") > _EXPERT_DOMAIN_STATUSES.index("audit-passed")
def test_mathematics_logic_composer_wired() -> None:
assert _EXPERT_COMPOSERS["mathematics_logic"] == "core.capability.expert_promotion_math"
# ---------------------------------------------------------------------------
# Path-stability fix (digest filesystem-independence)
# ---------------------------------------------------------------------------
def test_rel_returns_repo_relative_posix() -> None:
"""``_rel`` must return repo-relative POSIX, not absolute path,
so the digest is stable across operator filesystems."""
p = _REPO_ROOT / "evals" / "math_bounded_grammar" / "v1" / "cases.jsonl"
rel = _composer_rel(p)
assert not rel.startswith("/") # not absolute
assert rel == "evals/math_bounded_grammar/v1/cases.jsonl"
# composite_math_gate's _rel must agree
assert _gate_rel(p) == rel
def test_rel_falls_back_to_absolute_outside_repo(tmp_path: Path) -> None:
"""Paths outside the repo root should fall back to absolute
POSIX (rather than raise)."""
outside = tmp_path / "elsewhere.json"
rel = _composer_rel(outside)
assert rel == outside.as_posix()
def test_composer_digest_uses_relative_paths() -> None:
"""Sanity: the digest's input bundle should contain repo-relative
paths (not absolute), so it's filesystem-independent."""
v = evaluate_math_expert_promotion()
for o in v.obligations:
assert not o.evidence_pointer.startswith("/"), (
f"obligation {o.obligation_id} has absolute path: {o.evidence_pointer}"
)
def test_composite_gate_digest_uses_relative_paths() -> None:
v = evaluate_composite_math_gate()
for b in v.benchmarks:
assert not b.report_path.startswith("/"), (
f"benchmark {b.benchmark_id} has absolute report_path: {b.report_path}"
)
# ---------------------------------------------------------------------------
# Ledger flip — mathematics_logic must reach "expert" on current main
# ---------------------------------------------------------------------------
def _math_row(report: dict) -> dict:
for row in report["domains"]:
if row["domain"] == "mathematics_logic":
return row
raise AssertionError("mathematics_logic row missing from ledger")
def test_mathlogic_status_is_expert_with_signed_evidence() -> None:
"""The load-bearing snapshot. Given the signed entry in
docs/reviewers.yaml AND every obligation auditor passing AND the
composite gate passing, the ledger reports
``mathematics_logic.status == "expert"``."""
report = ledger_report()
row = _math_row(report)
assert row["status"] == "expert", (
f"expected status=expert; got {row['status']!r}; "
f"audit_passed_reason={row.get('audit_passed_reason')}; "
f"expert_reason={row.get('expert_reason')}"
)
assert row["predicates"]["expert"] is True
assert row["predicates"]["audit_passed"] is True
def test_mathlogic_row_carries_expert_reason() -> None:
report = ledger_report()
row = _math_row(report)
assert "expert_reason" in row
assert "admitted" in row["expert_reason"].lower()
def test_other_domains_have_expert_false_no_composer_wired() -> None:
"""Domains without an entry in _EXPERT_COMPOSERS keep expert=False
even when audit_passed=True. Currently only mathematics_logic
is wired."""
report = ledger_report()
for row in report["domains"]:
if row["domain"] == "mathematics_logic":
continue
# Other domains' expert predicate must be False.
assert row["predicates"]["expert"] is False, (
f"{row['domain']} unexpectedly reports expert=True"
)
# ---------------------------------------------------------------------------
# Failure mode: signature missing → status stays at audit-passed
# ---------------------------------------------------------------------------
def test_composer_refuses_without_signature(tmp_path: Path) -> None:
"""If the reviewers file omits the math_expert_claims entry,
promote_admitted stays False with the expected awaiting-signature
message."""
fake = tmp_path / "reviewers.yaml"
fake.write_text(yaml.safe_dump({
"schema_version": 1,
"reviewers": [
{
"reviewer_id": "shay-j",
"display_name": "test",
"role": "primary",
"domains": ["*"],
"review_scope": ["pack", "proposal", "chain", "eval"],
"provenance": "test",
}
],
EXPERT_CLAIMS_KEY: [],
}), encoding="utf-8")
v = evaluate_math_expert_promotion(reviewers_path=fake)
assert v.technical_pass is True # evidence still passes
assert v.promote_admitted is False
assert "awaiting reviewer signature" in v.refusal_reason