Commit graph

1 commit

Author SHA1 Message Date
Shay
696f62abdd feat: ADR-0113 rename expert-demoaudit-passed; reserve expert namespace (ADR-0114 GSM8K roadmap)
The word "expert" in the previous status name implied raw-capability parity
with frontier LLMs on the same benchmark — which the gate does NOT verify.
What the gate actually verifies is CORE *claim-shape compliance*:

  * signed digest (replay-reproducible from on-disk lane results)
  * replay determinism (same inputs → byte-equal trace_hash)
  * typed refusal (fabrication refused, not paraphrased)
  * exact recall (no ANN, no cosine, no attention bottleneck)
  * grounding-source provenance

These are claim shapes a transformer LLM cannot structurally produce
regardless of raw accuracy. A frontier LLM might score higher on the
same benchmark but cannot pass this contract.

Rename scope (semantics only, per ADR-0113):

  status string         "expert-demo"        → "audit-passed"
  predicate key         predicates.expert_demo → predicates.audit_passed
  reason key            expert_demo_reason   → audit_passed_reason
  YAML key              expert_demo_claims   → audit_passed_claims
  CLI command           core demo expert     → core demo audit-passed
  output dir            evals/expert_demos/  → evals/audit_passed/
  artifact filenames    expert_demo.{json,html} → audit_passed.{json,html}
  HTML title            CORE Expert-Demo: X  → CORE Audit-Passed: X

Internal Python identifiers (module/file/function/class names like
`expert_demo.py`, `evaluate_expert_demo`, `ExpertDemoClaim`,
`expert_demo_claim_for`) are deliberately kept to minimize churn. ADR
file titles (ADR-0106..0112) preserved as historical record.

`expert` namespace reserved for ADR-0114+: an actual capability tier
above `audit-passed` backed by a public benchmark with a stated
threshold. ADR-0114 proposes the first such target — GSM8K-math —
laying out a falsifiable 7-phase arc (parser → solver → verifier →
stepped-realizer → eval lane → first `expert` ledger tier promotion).

Tests: 184 directly-affected tests green (140 capability/expert-demo
suite + 34 demo/audit-tour + 10 correction-cue). Smoke suite 67/67.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 15:36:10 -07:00