Shay 696f62abdd feat: ADR-0113 rename expert-demo → audit-passed; reserve expert namespace (ADR-0114 GSM8K roadmap)

The word "expert" in the previous status name implied raw-capability parity
with frontier LLMs on the same benchmark — which the gate does NOT verify.
What the gate actually verifies is CORE *claim-shape compliance*:

  * signed digest (replay-reproducible from on-disk lane results)
  * replay determinism (same inputs → byte-equal trace_hash)
  * typed refusal (fabrication refused, not paraphrased)
  * exact recall (no ANN, no cosine, no attention bottleneck)
  * grounding-source provenance

These are claim shapes a transformer LLM cannot structurally produce
regardless of raw accuracy. A frontier LLM might score higher on the
same benchmark but cannot pass this contract.

Rename scope (semantics only, per ADR-0113):

  status string         "expert-demo"        → "audit-passed"
  predicate key         predicates.expert_demo → predicates.audit_passed
  reason key            expert_demo_reason   → audit_passed_reason
  YAML key              expert_demo_claims   → audit_passed_claims
  CLI command           core demo expert     → core demo audit-passed
  output dir            evals/expert_demos/  → evals/audit_passed/
  artifact filenames    expert_demo.{json,html} → audit_passed.{json,html}
  HTML title            CORE Expert-Demo: X  → CORE Audit-Passed: X

Internal Python identifiers (module/file/function/class names like
`expert_demo.py`, `evaluate_expert_demo`, `ExpertDemoClaim`,
`expert_demo_claim_for`) are deliberately kept to minimize churn. ADR
file titles (ADR-0106..0112) preserved as historical record.

`expert` namespace reserved for ADR-0114+: an actual capability tier
above `audit-passed` backed by a public benchmark with a stated
threshold. ADR-0114 proposes the first such target — GSM8K-math —
laying out a falsifiable 7-phase arc (parser → solver → verifier →
stepped-realizer → eval lane → first `expert` ledger tier promotion).

Tests: 184 directly-affected tests green (140 capability/expert-demo
suite + 34 demo/audit-tour + 10 correction-cue). Smoke suite 67/67.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-22 15:36:10 -07:00

1.3 KiB

Raw Blame History

Capability Reports

Phase A-C adds read-only capability reports. These commands do not mutate packs, execute dynamic validators, or flip dormant runtime flags.

Available commands:

core capability chains --json
core capability flags --json
core capability ledger --json
core capability artifact --lane cognition --split public --version v1 --json
core capability domain-contract --pack-id en_core_cognition_v1 --json
core capability evidence-plan --json

Day-1 output is expected to be low-status:

Domain	Expected status	Primary blocker
`systems_software`	`reasoning-capable`	audit-passed still gated by domain eval thresholds and replay/provenance evidence
`mathematics_logic`	`reasoning-capable`	audit-passed still gated by domain eval thresholds and replay/provenance evidence
`physics`	`reasoning-capable`	audit-passed still gated by domain eval thresholds and replay/provenance evidence
`hebrew_greek_textual_reasoning`	`reasoning-capable`	audit-passed still gated by eval thresholds and replay/provenance evidence
`philosophy_theology`	`reasoning-capable`	audit-passed still gated by eval thresholds and replay/provenance evidence

Status is generated from predicates. A blocked row lifts only when the named gap is closed and the next status predicate passes.

1.3 KiB Raw Blame History

Capability Reports

1.3 KiB

Raw Blame History