Closes the documentation audit. All four lower-priority surfaces from the earlier scan now reflect the post-ADR-0110 reality. New: - docs/sessions/SESSION-2026-05-22-contract-layer-arc.md: full narrative of the ADR-0103 -> ADR-0110 arc, including the refused/amended/succeeded narrative, infrastructure bridges, and ledger state at session close. Pairs with the 2026-05-21 articulation-arc session log. Extended (additive, no rewrites): - docs/runtime_contracts.md: new 'Expert-Demo Promotion Contract (ADR-0106 + ADR-0109)' section. Surfaces ledger_report() shape, reviewer-yaml schema, lane-shape registry, replay invariant, fail-closed registry behavior, and trust boundary. - docs/eval_methodology.md: new 'Lane-shape registry (ADR-0109)' section. Documents what to do when adding a new eval lane: pick a shape or amend ADR-0109. Notes the holdout-runner gating (cases_plaintext.jsonl vs sealed). Header bumped to 2026-05-22. - docs/Whitepaper.md: new section XIII 'Evidence-Governed Domain Layer' between XII (Forward Semantic Control) and the Extensions/closing. Narrates the contract chain at the philosophical / external-reader level. The original §XIII becomes §XIV. - docs/Yellowpaper.md: new section XII 'Ratification Contract (ADR-0091 + ADR-0106 + ADR-0109)'. Formal specification of the nine predicates, the promotion predicate (with explicit logical form), the digest function, and the lane-shape registry table. The original §XII becomes §XIII. No code, no tests, no contract changes.
This commit is contained in:
Shay
GitHub
parent
95d3b4a45a
commit
bc90622e53
5 changed files with 427 additions and 4 deletions
|
|
@ -357,7 +357,64 @@ characterization), `docs/evals/phase6_comparative_demo.md`
|
|||
|
||||
---
|
||||
|
||||
### XIII. Extensions
|
||||
### XIII. Evidence-Governed Domain Layer
|
||||
|
||||
Substrate alone does not make a claim credible. The Whitepaper's first
|
||||
twelve sections describe what CORE *is*. This section describes how the
|
||||
project distinguishes *what it has demonstrated* from *what its
|
||||
substrate makes possible* — a distinction that the architecture made
|
||||
visible only after the substrate stabilized.
|
||||
|
||||
The discipline is encoded as a chain of accepted Architecture
|
||||
Decision Records (ADR-0091 through ADR-0110):
|
||||
|
||||
1. **Contract.** ADR-0091 defines nine predicate checks every ratified
|
||||
pack must satisfy: lemma coverage, operator-chain counts, intent-shape
|
||||
coverage, holdout discipline, reviewer resolution, and others. A pack
|
||||
that satisfies all nine earns a `reasoning-capable` ledger row.
|
||||
2. **Trust root.** ADR-0092 introduces a YAML-anchored reviewer registry.
|
||||
Pack ratification cites a registered reviewer; the registry is the
|
||||
only trust root.
|
||||
3. **Fabrication control.** ADR-0096 defines a negative-control eval
|
||||
lane. Phantom endpoints, cross-pack non-bridges, and sibling
|
||||
collapses must all *refuse*. A pack that confabulates on any
|
||||
refusal class fails ratification.
|
||||
4. **Promotion gate.** ADR-0106 introduces the second status above
|
||||
`reasoning-capable`: `expert_demo`. Promotion requires a reviewer-
|
||||
signed `expert_demo_claims` entry whose evidence-bundle SHA-256
|
||||
reproduces byte-for-byte from on-disk lane results. The signer must
|
||||
resolve to the registry; the signed lanes must be attached to the
|
||||
domain's ratified packs.
|
||||
5. **Shape registry.** ADR-0109 amends ADR-0106 with explicit per-shape
|
||||
threshold rules (cognition, accuracy, inference, refusal,
|
||||
symbolic-logic). Unknown lanes fail closed.
|
||||
|
||||
The contract is load-bearing because it has *refused*. ADR-0107 records
|
||||
the first promotion attempt — `mathematics_logic` — being honestly
|
||||
refused on two named blockers. ADR-0109 amended the threshold rules
|
||||
without weakening the discipline. ADR-0110 then promoted
|
||||
`mathematics_logic` as the first domain at `expert_demo=true`, with the
|
||||
signed claim digest reproducing from disk.
|
||||
|
||||
The architectural commitment is: a system cannot claim to *do*
|
||||
something it has not been seen to do. A transformer LLM does not have
|
||||
a place to attach this commitment — its outputs are generated under
|
||||
sampling, not from a replay-deterministic trace bound to a signed
|
||||
evidence bundle. CORE makes the commitment first-class.
|
||||
|
||||
External readers can inspect the ledger
|
||||
(`core capability ledger` / `docs/decisions/README.md` table) to see
|
||||
which domains are *contract-passing* and which are *demonstrated*. As
|
||||
of acceptance: one domain demonstrated (`mathematics_logic`); three
|
||||
ratified domains pending their own promotion ADRs.
|
||||
|
||||
Full evidence chain: `docs/decisions/README.md` (index + frontier),
|
||||
`docs/decisions/ADR-0091-domain-pack-contract-v1.md` through
|
||||
`ADR-0110-mathematics-logic-expert-demo-promotion.md`.
|
||||
|
||||
---
|
||||
|
||||
### XIV. Extensions
|
||||
|
||||
**CORE-Logos** — The language articulation subsystem. Specified in the companion Yellow and White Addenda inherited from `core-ai`. The Logos defines the vocabulary manifold, the token projection law, the holonomy encoder, and the termination condition.
|
||||
|
||||
|
|
|
|||
|
|
@ -637,7 +637,106 @@ cargo test
|
|||
|
||||
---
|
||||
|
||||
### XII. What Was Deleted and Why
|
||||
### XII. Ratification Contract (ADR-0091 + ADR-0106 + ADR-0109)
|
||||
|
||||
The runtime contracts in §I–§XI describe the engine's algebraic
|
||||
behavior. The ratification contract describes the discipline under
|
||||
which the *capability ledger* is allowed to make claims about a
|
||||
domain.
|
||||
|
||||
#### Domain Pack Contract v1 (ADR-0091)
|
||||
|
||||
A pack manifest at `language_packs/data/<pack_id>/manifest.json`
|
||||
satisfies the contract iff all nine predicates hold:
|
||||
|
||||
1. **lemma_coverage** — declared lemmas resolve in `lexicon.jsonl`.
|
||||
2. **gloss_coverage_above_floor** — mount-eligible if gloss coverage
|
||||
crosses the per-pack floor.
|
||||
3. **operator_chain_count** — declared operator families each carry
|
||||
at least `_CHAINS_PER_OPERATOR_DOMAIN` chains.
|
||||
4. **intent_shape_coverage** — at least three intent shapes present.
|
||||
5. **holdout_present** — `evals/<lane>/holdouts/` exists with sealed
|
||||
or dev-mode-plaintext cases.
|
||||
6. **eval_lanes_uniform** — all packs in a multi-pack domain declare
|
||||
identical lane sets.
|
||||
7. **fabrication_control_passing** — phantom / cross-pack / sibling
|
||||
refusal classes all clean.
|
||||
8. **reviewer_resolution** — provenance reviewer id resolves in
|
||||
`docs/reviewers.yaml`.
|
||||
9. **deterministic_replay** — the canonical eval reports reproduce
|
||||
under `core test --suite cognition`.
|
||||
|
||||
A pack passing all nine earns `status = reasoning-capable` in the
|
||||
generated ledger row.
|
||||
|
||||
#### Expert-Demo Promotion (ADR-0106 + ADR-0109)
|
||||
|
||||
The promotion to `status = expert-demo` is contract-gated. The
|
||||
promotion predicate (`core/capability/expert_demo.py::evaluate_expert_demo`)
|
||||
requires:
|
||||
|
||||
```text
|
||||
reasoning_capable(D)
|
||||
∧ ∃ claim ∈ ReviewerRegistry.expert_demo_claims
|
||||
: claim.domain_id == D
|
||||
∧ ReviewerRegistry.can_review(claim.signed_by, D, scope="eval")
|
||||
∧ claim.evidence_lanes ⊆ ratified_lanes(D)
|
||||
∧ ∀ lane ∈ claim.evidence_lanes, split ∈ {public, holdout} :
|
||||
shape_checker(lane)(result(lane, v1, split))
|
||||
∧ derive_evidence_digest(D, claim.evidence_revision,
|
||||
claim.evidence_lanes, lane_results)
|
||||
== claim.claim_digest
|
||||
```
|
||||
|
||||
The digest function:
|
||||
|
||||
```text
|
||||
derive_evidence_digest(D, rev, lanes, results) =
|
||||
SHA-256(JSON.canonicalize({
|
||||
domain_id: D,
|
||||
evidence_revision: rev,
|
||||
evidence_lanes: sort(lanes),
|
||||
lane_metrics: {lane: {public: results[lane].public,
|
||||
holdout: results[lane].holdout}
|
||||
for lane in sort(lanes)}
|
||||
}))
|
||||
```
|
||||
|
||||
Canonicalization rules: sorted keys, compact separators,
|
||||
`ensure_ascii=False`. The same lane results must reproduce the same
|
||||
digest byte-for-byte; this is what makes the gate replay-deterministic.
|
||||
|
||||
#### Lane-Shape Registry (ADR-0109)
|
||||
|
||||
Threshold dispatch is per-lane-shape, not lane-uniform:
|
||||
|
||||
| Shape | Required keys | Pass condition |
|
||||
|---|---|---|
|
||||
| `cognition_shape` | `surface_groundedness`, `term_capture_rate`, `intent_accuracy`, `versor_closure_rate` | `≥ 0.95 ∧ ≥ 0.85 ∧ ≥ 0.95 ∧ == 1.0` |
|
||||
| `accuracy_shape` | `accuracy` *or* `(passed, total)` | `accuracy ≥ 0.95` (computed as `passed/total` if `accuracy` absent) |
|
||||
| `inference_shape` | `all_pass_rate`, `replay_determinism`, `overall_pass` | `≥ 0.95 ∧ == 1.0 ∧ True` |
|
||||
| `refusal_shape` | `by_class[*].n`, `.refused`, `.fabricated` | `∀ bucket: refused == n ∧ fabricated == 0` |
|
||||
| `symbolic_logic_shape` | `accuracy` | `≥ 0.95` |
|
||||
|
||||
Lane id → shape resolution is by registry lookup, not metric
|
||||
introspection. Unknown lanes fail closed.
|
||||
|
||||
#### Fail-Closed Discipline
|
||||
|
||||
- Unloadable reviewer registry → zero claims → no domain promotes.
|
||||
- Unregistered lane id → promotion fails with named reason.
|
||||
- Claim digest drift → promotion refused; ledger row demotes to
|
||||
`reasoning-capable`.
|
||||
- Signer outside `eval` scope → promotion refused.
|
||||
|
||||
This is the algebraic specification of the contract layer the
|
||||
Whitepaper §XIII narrates. The substrate makes both refusal and
|
||||
promotion first-class; the ratification contract makes the
|
||||
distinction visible to external readers.
|
||||
|
||||
---
|
||||
|
||||
### XIII. What Was Deleted and Why
|
||||
|
||||
The formal record is in `docs/DELETION_LOG.md`. The summary:
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
# Eval Methodology — Benchmark Discipline Contract
|
||||
|
||||
**Status:** Accepted (ADR-0016)
|
||||
**Last updated:** 2026-05-15
|
||||
**Status:** Accepted (ADR-0016, extended by ADR-0109)
|
||||
**Last updated:** 2026-05-22 (lane-shape registry section added)
|
||||
|
||||
This document defines the five rules that govern every eval lane in the CORE
|
||||
capability roadmap. No exceptions per phase. A lane that does not satisfy these
|
||||
|
|
@ -102,3 +102,61 @@ A lane without a `contract.md` does not run.
|
|||
|
||||
- ADR-0016: Capability Roadmap
|
||||
- `docs/capability_roadmap.md`: Full phased plan
|
||||
|
||||
---
|
||||
|
||||
## Lane-shape registry (ADR-0109)
|
||||
|
||||
ADR-0091's Domain Pack Contract v1 introduced a `dev/public/holdout`
|
||||
discipline that every ratified pack must declare. ADR-0106 added a
|
||||
reviewer-signed expert-demo promotion gate that consults those same
|
||||
lane outputs at the ledger level. ADR-0109 then formalized the rule
|
||||
that **threshold dispatch is lane-shape-aware**, not lane-uniform.
|
||||
|
||||
### What this means for new lanes
|
||||
|
||||
Adding a new eval lane requires deciding which shape it reports:
|
||||
|
||||
| Shape | Required metrics | Threshold |
|
||||
|---|---|---|
|
||||
| `cognition_shape` | `surface_groundedness`, `term_capture_rate`, `intent_accuracy`, `versor_closure_rate` | 0.95 / 0.85 / 0.95 / 1.0 |
|
||||
| `accuracy_shape` | `accuracy` (or `passed`/`total` fallback) | ≥ 0.95 |
|
||||
| `inference_shape` | `all_pass_rate`, `replay_determinism`, `overall_pass` | 0.95 / 1.0 / true |
|
||||
| `refusal_shape` | `by_class[*].n`, `by_class[*].refused`, `by_class[*].fabricated` | refused == n, fabricated == 0 |
|
||||
| `symbolic_logic_shape` | `accuracy` | ≥ 0.95 |
|
||||
|
||||
A lane that does not fit any existing shape **must not be silently
|
||||
broadened**. The path is:
|
||||
|
||||
1. Open an ADR amending ADR-0109 to add the new shape.
|
||||
2. Add the shape checker to `SHAPE_CHECKERS` in
|
||||
`core/capability/expert_demo.py`.
|
||||
3. Add the lane → shape mapping to `LANE_SHAPE_REGISTRY`.
|
||||
|
||||
A lane id absent from the registry is **fail-closed** at the
|
||||
expert-demo gate (reason:
|
||||
`lane <id> has no registered shape — introduce via ADR amendment`).
|
||||
Unregistered lanes can still run as exploration; they just cannot
|
||||
contribute evidence to a `reviewer-signed expert_demo` promotion.
|
||||
|
||||
### Holdout-runner gating (ADR-0105)
|
||||
|
||||
`split='holdout'` runs go through `evals.holdout_runner._decrypt_holdout`,
|
||||
which expects either:
|
||||
|
||||
- `holdouts/v1/cases.jsonl.age` (sealed, requires `CORE_HOLDOUT_KEY`), **or**
|
||||
- `holdouts/v1/cases_plaintext.jsonl` (dev-mode fallback, no key required).
|
||||
|
||||
A bare `holdouts/v1/cases.jsonl` is invisible to the runner. Lanes
|
||||
authored before ADR-0105 must either rename their plaintext file or
|
||||
seal it against an age recipient to be runnable on the `holdout`
|
||||
split.
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- ADR-0091: Domain Pack Contract v1 (`evals/<lane>/holdouts/` discipline)
|
||||
- ADR-0105: Sealed-holdout encryption (dev-mode fallback preserved)
|
||||
- ADR-0106: Expert-demo promotion contract (consumes lane results)
|
||||
- ADR-0109: Lane-shape-aware thresholds (this section's authority)
|
||||
|
|
|
|||
|
|
@ -341,3 +341,112 @@ Content-addressing rules (binding across the whole pipeline):
|
|||
|
||||
See `formation/hashing.py`, `formation/cache.py`, and `formation/forge.py`
|
||||
for the implementation of each rule.
|
||||
|
||||
---
|
||||
|
||||
## Expert-Demo Promotion Contract (ADR-0106 + ADR-0109)
|
||||
|
||||
Adds a domain-aware, reviewer-signed promotion gate to the capability
|
||||
ledger surface (`ledger_report()`). Distinct from the runtime/turn
|
||||
contracts above: this contract governs *what the ledger is allowed to
|
||||
claim about a domain*, not what the runtime does on any single turn.
|
||||
|
||||
Per ADR-0108, the contract has been demonstrated end-to-end —
|
||||
refused once (ADR-0107), amended once (ADR-0109), succeeded once
|
||||
(ADR-0110).
|
||||
|
||||
### Surface
|
||||
|
||||
`ledger_report()` returns a `domains` list. Each row carries:
|
||||
|
||||
```text
|
||||
status ∈ {blocked, seeded, grounded, reasoning-capable, expert-demo}
|
||||
predicates.expert_demo bool
|
||||
expert_demo_reason str (one-line legibility for operators)
|
||||
```
|
||||
|
||||
A row carries `expert_demo=True` iff **all** of:
|
||||
|
||||
1. `reasoning_capable == True` (the ADR-0091 nine-predicate gate).
|
||||
2. A signed `ExpertDemoClaim` exists in `docs/reviewers.yaml` for the
|
||||
domain.
|
||||
3. The signer named in `claim.signed_by` has `eval` scope for the
|
||||
domain per `ReviewerRegistry.can_review` (ADR-0092).
|
||||
4. Every lane in `claim.evidence_lanes` is attached to at least one of
|
||||
the domain's ratified packs (no cross-domain bleed).
|
||||
5. Every named lane's public + holdout metrics meet the threshold for
|
||||
that lane's registered shape (ADR-0109; see §"Lane-shape registry"
|
||||
below).
|
||||
6. The canonical evidence-bundle SHA-256 reproduces `claim.claim_digest`
|
||||
byte-for-byte.
|
||||
|
||||
Any failure leaves the row at `reasoning-capable` with
|
||||
`expert_demo_reason` populated.
|
||||
|
||||
### Schema
|
||||
|
||||
`docs/reviewers.yaml` additively gains an `expert_demo_claims` block:
|
||||
|
||||
```yaml
|
||||
expert_demo_claims:
|
||||
- domain_id: mathematics_logic
|
||||
evidence_lanes:
|
||||
- elementary_mathematics_ood
|
||||
- inference_closure
|
||||
- fabrication_control
|
||||
evidence_revision: "adr-0110:reviewed:2026-05-22"
|
||||
signed_by: shay-j
|
||||
claim_digest: "94d74781..."
|
||||
```
|
||||
|
||||
The block is optional; absence yields zero promotions. Schema
|
||||
validation is loud: unknown signer ids, malformed digests, and
|
||||
duplicate `domain_id` values are all rejected at load time
|
||||
(`core.capability.reviewers.load_reviewer_registry`).
|
||||
|
||||
### Lane-shape registry (ADR-0109)
|
||||
|
||||
Threshold rules dispatch by lane shape, not uniformly. Registry in
|
||||
`core/capability/expert_demo.py`:
|
||||
|
||||
| Lane id | Shape | Threshold |
|
||||
|---|---|---|
|
||||
| `cognition` | `cognition_shape` | surface_groundedness ≥ 0.95, term_capture_rate ≥ 0.85, intent_accuracy ≥ 0.95, versor_closure_rate == 1.0 |
|
||||
| `elementary_mathematics_ood` | `accuracy_shape` | accuracy ≥ 0.95 (passed/total fallback) |
|
||||
| `foundational_physics_ood` | `accuracy_shape` | same |
|
||||
| `symbolic_logic` | `symbolic_logic_shape` | same as `accuracy_shape` |
|
||||
| `hebrew_fluency` | `accuracy_shape` | same |
|
||||
| `koine_greek_fluency` | `accuracy_shape` | same |
|
||||
| `inference_closure` | `inference_shape` | all_pass_rate ≥ 0.95, replay_determinism == 1.0, overall_pass == True |
|
||||
| `fabrication_control` | `refusal_shape` | every `by_class` bucket: refused == n, fabricated == 0 |
|
||||
|
||||
**Unknown lane ids fail closed** with reason
|
||||
`lane <id> has no registered shape — introduce via ADR amendment`.
|
||||
Adding a lane to the expert-demo surface requires an explicit registry
|
||||
entry, which requires an ADR amendment.
|
||||
|
||||
### Replay invariant
|
||||
|
||||
`core.capability.expert_demo.derive_evidence_digest` is deterministic
|
||||
in field order (sorted keys, compact separators). Re-running it
|
||||
against the on-disk lane results at the same `evidence_revision`
|
||||
reproduces `claim_digest` byte-for-byte. Drift in any lane result
|
||||
demotes the row back to `reasoning-capable` until re-signed.
|
||||
|
||||
### Fail-closed registry
|
||||
|
||||
If `docs/reviewers.yaml` fails to parse,
|
||||
`reporting._load_registry_for_expert_demo` returns an empty registry
|
||||
(zero reviewers, zero claims) rather than raising. Every domain row
|
||||
falls back to `expert_demo=false`. A broken registry must never
|
||||
silently grant `expert_demo=true`.
|
||||
|
||||
### Trust boundary
|
||||
|
||||
- Pack mutation remains proposal-only (ADR-0029/0064 discipline).
|
||||
- A reviewer signature in `expert_demo_claims` does not authorize
|
||||
pack mutation — only a ledger-row promotion. The two paths remain
|
||||
separate.
|
||||
- `evidence_revision` may be a labeled string (e.g.
|
||||
`adr-0110:reviewed:2026-05-22`) or a raw git sha. The load-bearing
|
||||
invariant is replay byte-equality, not git-sha format.
|
||||
|
|
|
|||
100
docs/sessions/SESSION-2026-05-22-contract-layer-arc.md
Normal file
100
docs/sessions/SESSION-2026-05-22-contract-layer-arc.md
Normal file
|
|
@ -0,0 +1,100 @@
|
|||
# Session 2026-05-22 — Contract Layer Arc (ADR-0103 → ADR-0110)
|
||||
|
||||
**Date:** 2026-05-22
|
||||
**Owner:** Joshua Shay (`shay-j`)
|
||||
**Pair:** Claude (Opus 4.7)
|
||||
**Outcome:** Evidence-Governed Domain Layer demonstrated end-to-end; first `expert_demo` promotion landed.
|
||||
|
||||
---
|
||||
|
||||
## Arc summary
|
||||
|
||||
The session opened with three in-flight Codex-drafted PRs (ADR-0103 fluency-lane attachment, ADR-0104 curriculum-sourced proposals, ADR-0105 sealed-holdout encryption) needing reconciliation and merge. It closed with the entire ADR-0091 evidence-governed domain chain demonstrated end-to-end, the first reviewer-signed `expert_demo` promotion landed on `mathematics_logic`, and the docs backfilled to reflect the work.
|
||||
|
||||
Net merged: **15 PRs** (106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121).
|
||||
|
||||
### Chronological sequence
|
||||
|
||||
1. **Codex reconciliation (PRs #106 / #107 / #108).** Fixed stale `curriculum_loop_closure` SHA pin; corrected pyrage API misuse (`Identity.from_str`, not `from_file`); resolved README rebase conflicts. All three merged.
|
||||
2. **ADR-0106 — Expert-Demo Promotion Contract (proposed → accepted).** Domain-aware, reviewer-signed, replay-deterministic gate. Closes the cognition-lane bleed at `core/capability/reporting.py:418`. PRs #109 (proposed) + #113 (implementation).
|
||||
3. **ADR-0108 — Proposed-ADR sequencing (meta).** Made the post-ADR-0105 frontier ordering explicit and revisable. PRs #111 + #112.
|
||||
4. **ADR-0107 — `mathematics_logic` expert-demo deferred.** First worked attempt at the new contract; honestly refused on two named blockers (metric-shape uniformity assumption + `inference_closure` substantively failing at 40%). PR #114.
|
||||
5. **ADR-0109 — Lane-shape-aware threshold amendment.** Five shapes (`cognition_shape`, `accuracy_shape`, `inference_shape`, `refusal_shape`, `symbolic_logic_shape`), 8 lane ids in the registry, unknown lanes fail-closed; cognition-shape thresholds preserved bit-identical. PRs #115 + #116.
|
||||
6. **PR #117 — `_CORRECTION_CUE_PREFIX_RE` guard.** Investigation revealed `inference_closure` had regressed from 100% to 40% between 2026-05-17 and 2026-05-22; root cause was the declarative-relation regex swallowing "Actually X precedes Y." into VERIFICATION instead of CORRECTION. Premise-emit path is gated on CORRECTION, so non-`is` relations stopped producing `PackMutationProposal` records.
|
||||
7. **ADR-0110 — `mathematics_logic` expert-demo promotion.** Surfaced and bridged a third transition gap (sealed-holdout dev-mode fallback files missing; `fabrication_control` holdout cases absent; top-level `by_class` not folded into metrics). Signed `expert_demo_claims` entry added to `docs/reviewers.yaml` with digest `94d74781e103854230c1a71590e4df2287f5d2e87832f1c29b8ec4618853c04b`. **First domain at `expert_demo=true`.** PR #118.
|
||||
8. **ADR-0080 — Contemplation Loop Phase 1.** Delegated to Codex during ADR-0107/0109 investigation; merged in parallel as PR #119. Read-only frontier-compare miner, `SPECULATIVE`-only findings.
|
||||
9. **Documentation audit (#1 + #2).** PR #120 fixed the README broken `evals/CLAIMS.md` link and added an Evidence-Governed Domain Layer section. PR #121 backfilled `docs/PROGRESS.md` + `docs/capability_roadmap.md` with Phase 6 plus retroactive coverage of ADR-0027..0089.
|
||||
|
||||
---
|
||||
|
||||
## The contract demonstration narrative
|
||||
|
||||
ADR-0106's value is precisely that it can refuse. Two refusals happened in this session:
|
||||
|
||||
- **First refusal (ADR-0107).** The contract as written required cognition-pack-shape metrics uniformly across every attached lane. Math lanes produce `accuracy` / `all_pass_rate` / `by_class`, not `surface_groundedness`. The contract refused promotion by absence-of-key. This was the right answer; the wrong contract would have either silently failed-true or been mis-stretched to fit.
|
||||
- **Amendment (ADR-0109).** Lane-shape registry dispatches thresholds by shape. Unknown lanes fail-closed (introducing a new shape requires an ADR amendment, not a silent broadening). The amendment was small and self-contained because ADR-0106's other invariants (§1.1/§1.3/§1.4/§1.5) didn't need to move.
|
||||
- **First successful promotion (ADR-0110).** Math passes the now-amended contract honestly. Public + holdout meet thresholds across all three attached lanes; signed claim digest reproduces byte-for-byte; production ledger shows `mathematics_logic` at `status=expert-demo`.
|
||||
|
||||
The arc is intentionally visible. "ADR-0106 → refused → amended → succeeded" is the legibility upgrade.
|
||||
|
||||
---
|
||||
|
||||
## Infrastructure bridges landed under ADR-0110 (not contract changes)
|
||||
|
||||
ADR-0107 named two blockers; ADR-0110 surfaced a third — the sealed-holdout transition gap. Three small bridges:
|
||||
|
||||
1. `cases_plaintext.jsonl` dev-mode fallback copied into `evals/elementary_mathematics_ood/holdouts/v1/` and `evals/inference_closure/holdouts/v1/` (matches ADR-0105 dev-mode convention exactly).
|
||||
2. Authored 9 `fabrication_control` holdout cases across all three refusal classes (`phantom_endpoint`, `cross_pack_non_bridge`, `sibling_collapse`) — the lane's holdout file had previously been empty.
|
||||
3. `core/capability/reporting.py:_fetch_lane_split` folds top-level `by_class` into the metrics dict so `refusal_shape` sees a canonical layout regardless of lane-internal result-file shape.
|
||||
|
||||
These are not contract changes. ADR-0106 + ADR-0109 contract bodies are untouched.
|
||||
|
||||
---
|
||||
|
||||
## Tests / invariants pinned
|
||||
|
||||
- `tests/test_adr_0110_math_expert_demo.py` (4 cases): math at expert-demo, signed claim present, replay digest byte-equal, other domains unaffected.
|
||||
- `tests/test_lane_shape_thresholds.py` (13 cases): lane-shape explicit, shape thresholds named, unknown lane fails closed, cognition shape unchanged under amendment, plus dead-shape and threshold-value gates.
|
||||
- `tests/test_expert_demo_contract.py::TestProductionLedgerPromotionsAreSignedOnly`: rewritten from "no domain promoted" to "every promoted domain has a signed claim." Load-bearing invariant preserved.
|
||||
- `tests/test_correction_cue_prefix_routing.py` (10 cases): pin the intent-classifier regression fix from PR #117 in both directions.
|
||||
- `tests/test_adr_0107_deferral.py` retired (deferral resolved by ADR-0110).
|
||||
- `tests/test_contemplation_loop.py` (41 cases, Codex): pin Phase 1 read-only emission invariants.
|
||||
|
||||
---
|
||||
|
||||
## Ledger state at session close
|
||||
|
||||
| Domain | Status |
|
||||
|---|---|
|
||||
| `mathematics_logic` | **`expert-demo`** ✓ (ADR-0110) |
|
||||
| `physics` | `reasoning-capable` |
|
||||
| `systems_software` | `reasoning-capable` |
|
||||
| `hebrew_greek_textual_reasoning` | `reasoning-capable` |
|
||||
| `philosophy_theology` | `reasoning-capable` |
|
||||
|
||||
---
|
||||
|
||||
## What this session does NOT do
|
||||
|
||||
- Promote any domain other than `mathematics_logic`. The other three ratified domains are eligible under the now-amended contract but each needs its own signed claim + promotion ADR.
|
||||
- Implement ADR-0084 (definitional layer) or ADR-0087 (rhetorical style axis). Both remain Proposed per ADR-0108 sequencing.
|
||||
- Migrate from labeled `evidence_revision` (`adr-0110:reviewed:2026-05-22`) to raw git-sha form. The labeled form is acceptable under ADR-0106; tightening to git-sha-only would be a future ADR amendment.
|
||||
- Replace the plaintext holdout dev-mode fallback with sealed encryption. The dev-mode fallback is acceptable per ADR-0105's own §"Dev-mode fallback preserved" clause.
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- `docs/decisions/ADR-0103-fluency-lane-attachment-for-adr-0102.md`
|
||||
- `docs/decisions/ADR-0104-curriculum-sourced-teaching-proposals.md`
|
||||
- `docs/decisions/ADR-0105-sealed-holdout-encryption.md`
|
||||
- `docs/decisions/ADR-0106-expert-demo-promotion-contract.md`
|
||||
- `docs/decisions/ADR-0107-mathematics-logic-expert-demo-deferred.md`
|
||||
- `docs/decisions/ADR-0108-proposed-adr-sequencing.md`
|
||||
- `docs/decisions/ADR-0109-lane-shape-aware-thresholds.md`
|
||||
- `docs/decisions/ADR-0110-mathematics-logic-expert-demo-promotion.md`
|
||||
- `docs/decisions/ADR-0080-contemplation-loop.md`
|
||||
- `docs/reviewers.yaml` (first signed `expert_demo_claims` entry)
|
||||
- `core/capability/expert_demo.py`
|
||||
- `core/capability/reviewers.py`
|
||||
- `tests/test_adr_0110_math_expert_demo.py`
|
||||
Loading…
Reference in a new issue