PROGRESS.md was last updated mid-Phase-5 (~ADR-0021). The pack-layer chain (ADR-0027..0045), forward-graph+surface composer chain (ADR-0046..0089), and evidence-governed domain layer (ADR-0091..0110) were all absent. capability_roadmap.md was last updated 2026-05-17 and predated the expert-demo promotion contract entirely. Added to both: - Phase 6 — Evidence-Governed Domain Layer section covering ADR-0091 contract chain through ADR-0110 expert-demo arc, including the refused-amended-succeeded narrative and the current ledger state (math at expert-demo, four others at reasoning-capable). - Backfill subsections for ADR-0027..0045 (pack-layer chain) and ADR-0046..0089 (forward-graph + surface-composer + register + anchor-lens + contemplation-loop work) marked retrospective. capability_roadmap.md also gains a '2026-05-22' status update at the top alongside the existing 2026-05-17 entry. The full Phase 0-5 exit-criteria rewrite remains queued separately. No code, no tests, no claims changes.
This commit is contained in:
Shay
GitHub
parent
95e5e93d02
commit
95d3b4a45a
2 changed files with 156 additions and 2 deletions
|
|
@ -702,6 +702,99 @@ Per-surface bit-identity gates landed (2026-05-16):
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Phase 6 — Evidence-Governed Domain Layer
|
||||||
|
|
||||||
|
**Status:** IN PROGRESS (opened 2026-05-21, first promotion landed 2026-05-22)
|
||||||
|
**Depends on:** Phase 5 corpus flywheel + pack-layer chain (ADR-0027..0045)
|
||||||
|
**Roadmap:** see `docs/capability_roadmap.md` Phase 6 entry.
|
||||||
|
|
||||||
|
This phase ratifies the distinction between *contract-passing* (`reasoning-capable`) and *demonstrated* (`expert_demo=true`) at the capability ledger surface.
|
||||||
|
|
||||||
|
### Contract layer (all accepted)
|
||||||
|
|
||||||
|
- [x] **ADR-0091** Domain Pack Contract v1 — nine predicate checks on every ratified pack
|
||||||
|
- [x] **ADR-0092** Reviewer Registry v1 — schema-validated YAML reviewer roster
|
||||||
|
- [x] **ADR-0093** Domain Contract v1 implementation — runtime validator + ledger enforcement
|
||||||
|
- [x] **ADR-0094** Proposal source provenance — discriminated `ProposalSource(kind=...)`
|
||||||
|
- [x] **ADR-0095** Miner-sourced teaching proposals — `teaching/from_miner.py` + SHA-pinned `miner_loop_closure` lane
|
||||||
|
- [x] **ADR-0096** Fabrication-control eval lane — phantom / cross-pack / sibling-collapse refusals (SHA-pinned)
|
||||||
|
- [x] **ADR-0098** Demo composition contract
|
||||||
|
- [x] **ADR-0099** Public showcase demo (deterministic, byte-equal, <30s)
|
||||||
|
- [x] **ADR-0104** Curriculum-sourced teaching proposals — `teaching/from_curriculum.py` + SHA-pinned `curriculum_loop_closure` lane
|
||||||
|
- [x] **ADR-0105** Sealed-holdout encryption via age — dev-mode plaintext fallback preserved
|
||||||
|
|
||||||
|
### Reasoning-capable ratifications
|
||||||
|
|
||||||
|
- [x] **ADR-0097** `mathematics_logic` reasoning-capable (now superseded by ADR-0110 expert-demo, see below)
|
||||||
|
- [x] **ADR-0100** `physics` reasoning-capable
|
||||||
|
- [x] **ADR-0101** `systems_software` reasoning-capable
|
||||||
|
- [x] **ADR-0102** `hebrew_greek_textual_reasoning` reasoning-capable (first multi-pack ratification: 4 packs)
|
||||||
|
- [x] **ADR-0103** Hebrew + Koine Greek fluency lane attachment to ADR-0102 packs
|
||||||
|
|
||||||
|
### Expert-demo arc (the contract demonstrated end-to-end)
|
||||||
|
|
||||||
|
- [x] **ADR-0106** Expert-Demo Promotion Contract — domain-aware, reviewer-signed, replay-deterministic
|
||||||
|
- [x] **ADR-0107** `mathematics_logic` expert-demo deferred — first promotion attempt **honestly refused** on two named blockers (metric-shape uniformity; `inference_closure` 40% pass)
|
||||||
|
- [x] **ADR-0108** Proposed-ADR sequencing — meta-decision pinning the post-ADR-0105 frontier
|
||||||
|
- [x] **ADR-0109** Lane-shape-aware threshold amendment — 8 lane ids → 5 shapes (`cognition_shape`, `accuracy_shape`, `inference_shape`, `refusal_shape`, `symbolic_logic_shape`); unknown lanes fail-closed; cognition-shape thresholds preserved bit-identical
|
||||||
|
- [x] PR #117 — fix intent-classifier regression that had broken `inference_closure` (`_CORRECTION_CUE_PREFIX_RE` guard)
|
||||||
|
- [x] **ADR-0110** `mathematics_logic` expert-demo promoted — **first domain at `expert_demo=true`** in project history; signed claim digest reproduces byte-for-byte from on-disk lane results
|
||||||
|
|
||||||
|
### Contract demonstration narrative
|
||||||
|
|
||||||
|
The ADR-0106 contract refused once (0107), amended once cleanly (0109), and succeeded once (0110). External readers can now distinguish the two ceilings (`reasoning-capable` vs `expert-demo`) by inspecting the ledger.
|
||||||
|
|
||||||
|
### Current ledger state (per `core capability ledger`)
|
||||||
|
|
||||||
|
| Domain | Status |
|
||||||
|
|---|---|
|
||||||
|
| `mathematics_logic` | **`expert-demo`** ✓ |
|
||||||
|
| `physics` | `reasoning-capable` |
|
||||||
|
| `systems_software` | `reasoning-capable` |
|
||||||
|
| `hebrew_greek_textual_reasoning` | `reasoning-capable` |
|
||||||
|
| `philosophy_theology` | `reasoning-capable` |
|
||||||
|
|
||||||
|
### Open within Phase 6
|
||||||
|
|
||||||
|
- [ ] Second expert-demo promotion (any of `physics` / `systems_software` / `hebrew_greek_textual_reasoning` — all now eligible under ADR-0109 shape rules)
|
||||||
|
- [ ] Multi-reviewer threshold signing (open candidate frontier item from ADR-0105)
|
||||||
|
|
||||||
|
### Pack-layer chain — ADR-0027 through ADR-0045 (backfill)
|
||||||
|
|
||||||
|
The pack-layer architecture that Phase 6 builds on was established as a sequence between Phases 4 and 5 work but never registered here. Captured retroactively:
|
||||||
|
|
||||||
|
- [x] **ADR-0027** Identity packs — `IdentityManifold` loaded from swappable, content-addressed pack
|
||||||
|
- [x] **ADR-0028** Identity surface wiring — packs carry `surface_preferences` consumed by assembler
|
||||||
|
- [x] **ADR-0029** Safety packs — sibling to identity, never-swappable, fail-closed at startup
|
||||||
|
- [x] **ADR-0033** Ethics packs — third pack layer; swappable like identity, propositional like safety
|
||||||
|
- [x] **ADR-0035** Turn-loop verdicts — `SafetyCheck` + `EthicsCheck` auto-invoked
|
||||||
|
- [x] **ADR-0036** Safety refusal policy — typed runtime refusal
|
||||||
|
- [x] **ADR-0037** Ethics refusal opt-in
|
||||||
|
- [x] **ADR-0038** Hedge injection — soft remediation
|
||||||
|
- [x] **ADR-0039** Audit completeness — `TurnVerdicts` bundle + stub-path emission
|
||||||
|
- [x] **ADR-0040** Telemetry sink — structured JSONL turn-event sink
|
||||||
|
- [x] **ADR-0041** CLI verdicts + fan-out
|
||||||
|
- [x] **ADR-0042** Audit-tour demo — `core demo audit-tour`
|
||||||
|
- [x] **ADR-0043** Pack measurements — identity-divergence + refusal-calibration runners
|
||||||
|
- [x] **ADR-0044** Medical-ethics pack — worked-example domain ethics pack
|
||||||
|
- [x] **ADR-0045** Long-context comparison evidence
|
||||||
|
|
||||||
|
### Forward-graph + surface-composer chain — ADR-0046 through ADR-0089 (backfill)
|
||||||
|
|
||||||
|
Also retroactive (the bulk happened between 2026-05-18 and 2026-05-20):
|
||||||
|
|
||||||
|
- [x] **ADR-0046/0047** Forward graph constraint — PropositionGraph → AdmissibilityRegion *before* generate runs
|
||||||
|
- [x] **ADR-0048..0066** Pack-grounded surface composers for every intent shape (DEFINITION, RECALL, COMPARISON, PROCEDURE, CORRECTION, NARRATIVE, EXAMPLE)
|
||||||
|
- [x] **ADR-0063/0064** Cross-pack resolver + cross-pack teaching corpora
|
||||||
|
- [x] **ADR-0068..0072** Register substrate (terse / convivial / formal); orthogonal axis — register holds `trace_hash` CONSTANT
|
||||||
|
- [x] **ADR-0073** Anchor lens substrate — substantive variation axis; **opposite** invariant from register (lens moves `trace_hash` DISTINCT)
|
||||||
|
- [x] **ADR-0078** Composer/graph atom equivalence telemetry
|
||||||
|
- [x] **ADR-0080** Contemplation Loop Phase 1 — read-only frontier-compare miner, `SPECULATIVE`-only findings (landed 2026-05-22)
|
||||||
|
- [x] **ADR-0083** Transitive chain surface — bounded multi-hop teaching-grounded surface
|
||||||
|
- [x] **ADR-0089** Discourse planner + compound-intent dispatch
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Open Scope Decisions
|
## Open Scope Decisions
|
||||||
|
|
||||||
| Decision | Status | Deadline |
|
| Decision | Status | Deadline |
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,7 @@
|
||||||
|
|
||||||
**Status:** Draft, derived from `docs/sessions/SESSION-2026-05-15-capability-gates.md`
|
**Status:** Draft, derived from `docs/sessions/SESSION-2026-05-15-capability-gates.md`
|
||||||
**Owner:** Joshua Shay
|
**Owner:** Joshua Shay
|
||||||
**Last updated:** 2026-05-17
|
**Last updated:** 2026-05-22 (Phase 6 added; status update appended below)
|
||||||
|
|
||||||
## Status update — 2026-05-17
|
## Status update — 2026-05-17
|
||||||
|
|
||||||
|
|
@ -35,7 +35,22 @@ Closed gates (vs the Phase 1 "Foundational Triple"):
|
||||||
A full rewrite of the phase exit-criteria against current state is queued for
|
A full rewrite of the phase exit-criteria against current state is queued for
|
||||||
the next planning pass.
|
the next planning pass.
|
||||||
|
|
||||||
This document walks CORE from its present state through the gating framework defined in the 2026-05-15 session. It is organized into six phases. Each phase has entry criteria, work items, exit criteria, and a benchmark discipline contract.
|
## Status update — 2026-05-22
|
||||||
|
|
||||||
|
Phase 5 (Curriculum Era) work landed in volume during the 2026-05-17 → 2026-05-22 window. The roadmap gains a new Part-II phase (**Phase 6 — Evidence-Governed Domain Layer**, see below) that did not exist in the 2026-05-15 draft because the substrate it ratifies (multi-domain packs, signed reviewer registry, expert-demo gate) did not yet exist either. The phase is documented here as accepted and partially landed, not proposed.
|
||||||
|
|
||||||
|
Major chains accepted since 2026-05-17:
|
||||||
|
|
||||||
|
* **Pack-layer chain — ADR-0027 through ADR-0045.** Identity packs (0027/0028), safety packs (0029), ethics packs (0033/0036/0037), reviewed verdicts (0035), audit completeness (0039), telemetry sink (0040), CLI verdicts + fan-out (0041), audit-tour demo (0042), pack measurements (0043), medical-ethics worked example (0044), long-context comparison evidence (0045).
|
||||||
|
* **Forward graph constraint → pack-grounded surface — ADR-0046 through ADR-0070.** PropositionGraph becomes AdmissibilityRegion before generate (0046/0047); pack-grounded surface composers for DEFINITION/RECALL/COMPARISON/PROCEDURE/CORRECTION/NARRATIVE/EXAMPLE intents (0048–0066); cross-pack resolver and teaching-corpus chain (0063/0064); register terse v1 (0070); seeded variation (0071); register telemetry + tour (0072).
|
||||||
|
* **Anchor lens substrate — ADR-0073 + sub-ADRs.** Substantive variation axis sibling to register, with the opposite invariant (lens moves trace_hash DISTINCT; register holds it CONSTANT). Both orthogonal-axis demos CI-pinned simultaneously.
|
||||||
|
* **Transitive chain surface and definitional groundwork — ADR-0078–0089.** Composer/graph atom equivalence telemetry (0078); transitive chain surface (0083); compound-intent dispatch + discourse planner (0089).
|
||||||
|
* **Contemplation Loop Phase 1 — ADR-0080 (accepted 2026-05-22).** Read-only frontier-compare miner emits `SPECULATIVE` findings only; routes through `teaching/review.py`; no pack mutation. The first system-emitted gap-finding surface.
|
||||||
|
* **Evidence-governed domain layer — ADR-0091 through ADR-0110.** See Phase 6 below.
|
||||||
|
|
||||||
|
A full Phase 0–5 exit-criteria rewrite against this expanded state remains queued; that pass is independent of the Phase 6 documentation.
|
||||||
|
|
||||||
|
This document walks CORE from its present state through the gating framework defined in the 2026-05-15 session. It is organized into six phases (now seven with Phase 6 below). Each phase has entry criteria, work items, exit criteria, and a benchmark discipline contract.
|
||||||
|
|
||||||
The benchmark discipline is the spine of the plan. Without it, the phases become aspirational. With it, "are we there yet" becomes a CLI question.
|
The benchmark discipline is the spine of the plan. Without it, the phases become aspirational. With it, "are we there yet" becomes a CLI question.
|
||||||
|
|
||||||
|
|
@ -323,6 +338,52 @@ The phase has no single exit criterion. Instead, each domain becomes its own sub
|
||||||
|
|
||||||
**This phase has no estimated duration.** It is the phase the project lives in after the engineering era ends. Frontier-LLM parity on breadth happens *inside* this phase if it happens at all — likely measured in years, not weeks, and at whatever sample efficiency Phase 4 demonstrated.
|
**This phase has no estimated duration.** It is the phase the project lives in after the engineering era ends. Frontier-LLM parity on breadth happens *inside* this phase if it happens at all — likely measured in years, not weeks, and at whatever sample efficiency Phase 4 demonstrated.
|
||||||
|
|
||||||
|
### Phase 6 — Evidence-Governed Domain Layer
|
||||||
|
|
||||||
|
**Entry criteria:** Phase 5 corpus flywheel operational (curriculum + miner sourcing actively producing reviewed proposals); pack-layer chain (ADR-0027..0045) closed; forward-graph-constraint and surface-composer chains (ADR-0046..0089) shipping.
|
||||||
|
|
||||||
|
**Goal.** Distinguish *contract-passing* from *demonstrated* at the ledger surface. A pack that satisfies the nine ADR-0091 predicates earns a `reasoning-capable` ledger row; promotion to `expert_demo=true` requires a reviewer-signed evidence-bundle digest that reproduces byte-for-byte from on-disk lane results.
|
||||||
|
|
||||||
|
**Substrate (all accepted as of 2026-05-22):**
|
||||||
|
|
||||||
|
| Layer | What it ratifies | ADR |
|
||||||
|
|---|---|---|
|
||||||
|
| Domain Pack Contract v1 | Nine predicate checks on every ratified pack. | 0091 |
|
||||||
|
| Reviewer Registry v1 | Schema-validated reviewer roster; primary vs. domain-scoped. | 0092 |
|
||||||
|
| Domain Contract v1 implementation | Validator + ledger enforcement at runtime. | 0093 |
|
||||||
|
| Proposal source provenance | Discriminated `ProposalSource(kind=...)` for every teaching proposal. | 0094 |
|
||||||
|
| Miner-sourced proposals | `teaching/from_miner.py`; SHA-pinned `miner_loop_closure` lane. | 0095 |
|
||||||
|
| Fabrication-control eval lane | Phantom / cross-pack / sibling-collapse refusals must clean. | 0096 |
|
||||||
|
| Demo composition contract | Demos compose from shipped modules; no parallel mechanism. | 0098 |
|
||||||
|
| Public showcase demo | Deterministic, byte-equal, <30s. | 0099 |
|
||||||
|
| Reasoning-capable ratifications | `mathematics_logic` (0097), `physics` (0100), `systems_software` (0101), `hebrew_greek_textual_reasoning` (0102). |
|
||||||
|
| Fluency lane attachment | Hebrew + Koine Greek lanes attached to ADR-0102 packs with dev/public/holdout. | 0103 |
|
||||||
|
| Curriculum-sourced proposals | `teaching/from_curriculum.py`; SHA-pinned `curriculum_loop_closure` lane. | 0104 |
|
||||||
|
| Sealed-holdout encryption | age-based; dev-mode plaintext fallback preserved. | 0105 |
|
||||||
|
| **Expert-demo promotion contract** | **Domain-aware, reviewer-signed, replay-deterministic gate.** | **0106** |
|
||||||
|
| Lane-shape registry | 8 lane ids dispatch to 5 shapes; unknown lanes fail-closed. | 0109 (amends 0106) |
|
||||||
|
|
||||||
|
**Worked promotion narrative.** The contract has been demonstrated end-to-end:
|
||||||
|
|
||||||
|
1. **ADR-0107** — first promotion attempt (`mathematics_logic`) honestly refused by the contract on two named blockers (metric-shape uniformity assumption; `inference_closure` substantively failing at 40% pass).
|
||||||
|
2. **ADR-0109** — threshold rules amended with explicit lane-shape registry; cognition-shape thresholds preserved bit-identical; four new shapes added (`accuracy_shape`, `inference_shape`, `refusal_shape`, `symbolic_logic_shape`); unknown lanes fail-closed.
|
||||||
|
3. PR #117 fixed the intent-classifier regression that had broken `inference_closure`.
|
||||||
|
4. **ADR-0110** — `mathematics_logic` promoted to `expert_demo=true` under the amended contract. Signed claim digest reproduces from on-disk lane results; first domain at expert-demo.
|
||||||
|
|
||||||
|
**Exit criteria (cumulative; each can land independently of the others):**
|
||||||
|
|
||||||
|
- ☑ Contract definition (ADR-0091..0093) and reviewer trust root (ADR-0092) accepted.
|
||||||
|
- ☑ Negative-control fabrication lane (ADR-0096) ratified and SHA-pinned.
|
||||||
|
- ☑ Four reasoning-capable domain ratifications (ADR-0097/0100/0101/0102).
|
||||||
|
- ☑ Expert-demo promotion contract accepted (ADR-0106 + ADR-0109 amendment).
|
||||||
|
- ☑ First worked promotion lands (ADR-0110 — `mathematics_logic`).
|
||||||
|
- ☐ Second worked promotion lands (next domain: TBD; `physics` / `systems_software` / `hebrew_greek_textual_reasoning` all eligible under the now-amended contract).
|
||||||
|
- ☐ Multi-reviewer signing (currently single-recipient; the open candidate frontier item from ADR-0105).
|
||||||
|
|
||||||
|
**Why this is its own phase.** Phases 1–5 measure capability *internally*. Phase 6 is the first layer that measures *what the system has actually demonstrated to an external reader* and forces the ledger to distinguish the two. Every prior phase makes claims about the substrate; Phase 6 makes the substrate's claims auditable.
|
||||||
|
|
||||||
|
**This phase has no estimated duration.** Each new domain promotion is a discrete unit of work; the contract is the durable artifact.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Part III — Cross-Cutting Considerations
|
## Part III — Cross-Cutting Considerations
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue