696f62abdd
The word "expert" in the previous status name implied raw-capability parity
with frontier LLMs on the same benchmark — which the gate does NOT verify.
What the gate actually verifies is CORE *claim-shape compliance*:
* signed digest (replay-reproducible from on-disk lane results)
* replay determinism (same inputs → byte-equal trace_hash)
* typed refusal (fabrication refused, not paraphrased)
* exact recall (no ANN, no cosine, no attention bottleneck)
* grounding-source provenance
These are claim shapes a transformer LLM cannot structurally produce
regardless of raw accuracy. A frontier LLM might score higher on the
same benchmark but cannot pass this contract.
Rename scope (semantics only, per ADR-0113):
status string "expert-demo" → "audit-passed"
predicate key predicates.expert_demo → predicates.audit_passed
reason key expert_demo_reason → audit_passed_reason
YAML key expert_demo_claims → audit_passed_claims
CLI command core demo expert → core demo audit-passed
output dir evals/expert_demos/ → evals/audit_passed/
artifact filenames expert_demo.{json,html} → audit_passed.{json,html}
HTML title CORE Expert-Demo: X → CORE Audit-Passed: X
Internal Python identifiers (module/file/function/class names like
`expert_demo.py`, `evaluate_expert_demo`, `ExpertDemoClaim`,
`expert_demo_claim_for`) are deliberately kept to minimize churn. ADR
file titles (ADR-0106..0112) preserved as historical record.
`expert` namespace reserved for ADR-0114+: an actual capability tier
above `audit-passed` backed by a public benchmark with a stated
threshold. ADR-0114 proposes the first such target — GSM8K-math —
laying out a falsifiable 7-phase arc (parser → solver → verifier →
stepped-realizer → eval lane → first `expert` ledger tier promotion).
Tests: 184 directly-affected tests green (140 capability/expert-demo
suite + 34 demo/audit-tour + 10 correction-cue). Smoke suite 67/67.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
3.1 KiB
3.1 KiB
CLAIMS
Every row below is mechanically derived from in-tree state and
verified by CI. Tier 1 rows come from core.capability.ledger_report;
Tier 2 rows come from scripts/verify_lane_shas.py's pinned SHAs.
Tier 1 — Ratified domains
Each row asserts: the domain's packs pass all nine ADR-0091 contract
predicates, declared operator chains meet the ≥8 / ≥3-intent floor,
the reviewer registry resolves the primary reviewer, and the ledger
status predicate evaluates to reasoning-capable with no open gaps.
| Domain | Status | ADR | Packs | Open gaps |
|---|---|---|---|---|
philosophy_theology |
reasoning-capable | ADR-0085 | 2 | 0 |
mathematics_logic |
audit-passed | ADR-0097 | 1 | 0 |
physics |
audit-passed | ADR-0100 | 1 | 0 |
systems_software |
reasoning-capable | ADR-0101 | 1 | 0 |
hebrew_greek_textual_reasoning |
reasoning-capable | ADR-0102 | 4 | 0 |
Tier 2 — Pinned lane reports
Each row asserts: running the lane's runner produces a JSON
report whose SHA-256 matches the pinned value below. Mismatch
is a CI failure (.github/workflows/lane-shas.yml).
| ADR | Lane | Purpose | Report path | Pinned SHA-256 |
|---|---|---|---|---|
| ADR-0092 | reviewer_registry |
Reviewer registry schema validates + bootstrap entry self-seals | evals/reviewer_registry/results/v1_dev.json |
681a2aab5aa4ffd58cd837ce5673c8b2a9545b570117aec3c02726a12f6876e6 |
| ADR-0093 | domain_contract_validation |
All ratified packs satisfy the 9 ADR-0091 contract predicates | evals/domain_contract_validation/results/v1_dev.json |
f9c06cdeea8fb36a0d3c320007618c3afc92d67702ef31bd36ebd9ae9ced473f |
| ADR-0095 | miner_loop_closure |
Miner-sourced proposals route through single reviewed teaching path | evals/miner_loop_closure/results/v1_dev.json |
9f071733abe7dcacf759f928548ce738fb639af3fd6e4c621a651b306d7e77ce |
| ADR-0096 | fabrication_control_summary |
Phantom endpoints / cross-pack non-bridges / sibling collapses refuse | evals/fabrication_control/results/v1_summary.json |
01e1b6b711141f2b4a14551d7df3ea482d8d6dd7b364a25c509f4f8d08cda8a8 |
| ADR-0098 | demo_composition |
Demos compose from shipped modules; no parallel mechanism | evals/demo_composition/results/v1_dev.json |
27d838241bf3ed9e15d0e918ec6d89a823494d7e17c2dab9777825af7188f20f |
| ADR-0099 | public_demo |
Public showcase runs deterministically under 30s; all claims supported | evals/public_demo/results/v1_dev.json |
4be6f47509435a24984713acfcebd88e61f4e1278096fa5dc88a09e8af2f87ba |
| ADR-0104 | curriculum_loop_closure |
Curriculum-sourced proposals route through single reviewed teaching path | evals/curriculum_loop_closure/results/v1_dev.json |
b46d56b2d209172cc3ffaf3776dc8dcfe55093f13587c5cb67372be6dfa23e8d |
Verification
python3 scripts/verify_lane_shas.py # verify all Tier 2 SHAs
core test --suite full -q # exercises Tier 1 invariants