Commit graph

2 commits

Author SHA1 Message Date
Shay
bc90622e53
docs: runtime_contracts + eval_methodology + papers + session log (audit #3) (#122)
Closes the documentation audit. All four lower-priority surfaces from
the earlier scan now reflect the post-ADR-0110 reality.

New:
- docs/sessions/SESSION-2026-05-22-contract-layer-arc.md: full
  narrative of the ADR-0103 -> ADR-0110 arc, including the
  refused/amended/succeeded narrative, infrastructure bridges, and
  ledger state at session close. Pairs with the 2026-05-21
  articulation-arc session log.

Extended (additive, no rewrites):
- docs/runtime_contracts.md: new 'Expert-Demo Promotion Contract
  (ADR-0106 + ADR-0109)' section. Surfaces ledger_report() shape,
  reviewer-yaml schema, lane-shape registry, replay invariant,
  fail-closed registry behavior, and trust boundary.
- docs/eval_methodology.md: new 'Lane-shape registry (ADR-0109)'
  section. Documents what to do when adding a new eval lane:
  pick a shape or amend ADR-0109. Notes the holdout-runner gating
  (cases_plaintext.jsonl vs sealed). Header bumped to 2026-05-22.
- docs/Whitepaper.md: new section XIII 'Evidence-Governed Domain
  Layer' between XII (Forward Semantic Control) and the
  Extensions/closing. Narrates the contract chain at the
  philosophical / external-reader level. The original §XIII
  becomes §XIV.
- docs/Yellowpaper.md: new section XII 'Ratification Contract
  (ADR-0091 + ADR-0106 + ADR-0109)'. Formal specification of the
  nine predicates, the promotion predicate (with explicit logical
  form), the digest function, and the lane-shape registry table.
  The original §XII becomes §XIII.

No code, no tests, no contract changes.
2026-05-22 13:28:03 -07:00
Shay
1e01f7794e feat(evals): Phase 0 — benchmark methodology lock-in and eval framework
Implement the eval infrastructure defined in ADR-0016 before building new
eval lanes. This establishes the discipline that governs the entire
capability roadmap.

- Generic eval framework (evals/framework.py): lane discovery, versioned
  scoring, result persistence
- Cognition lane retrofitted into new convention: 45 cases split into
  stratified dev (13) / public v1 (13) / holdout (19) sets with contract,
  runner, and recorded results
- Generalized `core eval <lane>` CLI: dynamic lane discovery, --list,
  --version, --split, --save, --json flags
- Holdout runner scaffold: plaintext fallback, encryption interface ready
- Baseline runner scaffold: pluggable frontier model interface
- Fix: CognitiveTurnPipeline.run() crashed on turn_log[-1] when the
  unknown-domain gate returned a stub without appending to turn_log
- ADR-0016, eval_methodology.md, PROGRESS.md, capability gates session log

Phase 0 exit audit found two methodology issues:
1. Pipeline turn_log crash (fixed here)
2. Versor drift in multi-turn sessions (pre-existing, under investigation)
2026-05-15 22:36:53 -07:00