From 10457bec69ca8c0d536857b309c81e8639d9827f Mon Sep 17 00:00:00 2001 From: Shay Date: Thu, 21 May 2026 17:34:09 -0700 Subject: [PATCH] =?UTF-8?q?docs(adr):=20propose=20ADRs=200092-0099=20?= =?UTF-8?q?=E2=80=94=20scale-up=20slate=20through=20public=20showcase?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Eight load-bearing ADRs closing the loop from contemplation Phase 5 through a public showcase demo. Each one is small and evidence-bearing; together they sequence the next arc without duplicating existing substrate. - 0092 Reviewer Registry v1 — populates docs/reviewers.yaml schema; unblocks all reasoning-capable claims under ADR-0091. - 0093 Domain Pack Contract v1 Implementation — wires ADR-0091's five follow-up items (parser, dry-run validator, chain registry, eval lane refs, reviewer resolution) so manifest fields actually gate status. - 0094 Proposal Source Provenance — sealed ProposalSource type widening proposal schemas ahead of 0095. - 0095 Miner-Sourced Teaching Proposals — closes the contemplation loop: articulation_quality / contradiction_detection / frontier_compare miners emit PackMutationProposal candidates routed through the single reviewed teaching path; identity-pack defense at construction, not review; replay-equivalence pre-gate. - 0096 Fabrication-Control Eval Lane — first negative-control measure; three case classes (phantom endpoint, cross-pack non-bridge, sibling collapse) with frozen thresholds (fabrication_rate ≤ 0.01). - 0097 Mathematics-Logic Reasoning-Capable Ratification — first domain claim under ADR-0091; chain corpus + eval lanes already exist, this is the formal contract ratification. - 0098 Demo Composition Contract — DemoCommand protocol so demos can be composed without reimplementation; deterministic JSON, no global state mutation, declared output paths only. - 0099 Public Showcase Demo — composes four scenes (determinism / honest unknown / reviewed learning / multi-hop+trace) under 30s; pure composition enforced by grep gate; JSON byte-equality CI-pinned. Landing order: 0092 → 0094 → 0095 → 0093 → 0096 → 0097 → 0098 → 0099. Deliberately not included: curriculum compiler, formation course runner, calculator operators, response-mode taxonomy expansion, learning-scale 10k harness. Each is deferred with a documented reason. --- .../ADR-0092-reviewer-registry-v1.md | 129 +++++++++++++++ ...-domain-pack-contract-v1-implementation.md | 129 +++++++++++++++ .../ADR-0094-proposal-source-provenance.md | 125 ++++++++++++++ ...R-0095-miner-sourced-teaching-proposals.md | 129 +++++++++++++++ .../ADR-0096-fabrication-control-eval-lane.md | 118 +++++++++++++ ...cs-logic-reasoning-capable-ratification.md | 155 +++++++++++++++++ .../ADR-0098-demo-composition-contract.md | 152 +++++++++++++++++ .../ADR-0099-public-showcase-demo.md | 156 ++++++++++++++++++ 8 files changed, 1093 insertions(+) create mode 100644 docs/decisions/ADR-0092-reviewer-registry-v1.md create mode 100644 docs/decisions/ADR-0093-domain-pack-contract-v1-implementation.md create mode 100644 docs/decisions/ADR-0094-proposal-source-provenance.md create mode 100644 docs/decisions/ADR-0095-miner-sourced-teaching-proposals.md create mode 100644 docs/decisions/ADR-0096-fabrication-control-eval-lane.md create mode 100644 docs/decisions/ADR-0097-mathematics-logic-reasoning-capable-ratification.md create mode 100644 docs/decisions/ADR-0098-demo-composition-contract.md create mode 100644 docs/decisions/ADR-0099-public-showcase-demo.md diff --git a/docs/decisions/ADR-0092-reviewer-registry-v1.md b/docs/decisions/ADR-0092-reviewer-registry-v1.md new file mode 100644 index 00000000..3e34b89a --- /dev/null +++ b/docs/decisions/ADR-0092-reviewer-registry-v1.md @@ -0,0 +1,129 @@ +# ADR-0092 — Reviewer Registry v1 + +**Status:** Proposed +**Date:** 2026-05-21 +**Author:** CORE agents + reviewers +**Depends on:** ADR-0091 + +--- + +## Context + +ADR-0091 defines Domain Pack Contract v1. Its validation semantics require +that every `reviewers` entry on a reasoning-capable pack resolve through +`docs/reviewers.yaml`. The file exists but is currently empty: + +```yaml +reviewers: [] +``` + +This means no pack can advance beyond `grounded` under the contract, no +matter how complete its chains, evals, or gloss coverage. The empty +registry is a hard block on every subsequent ADR that wants to ratify a +domain claim. + +The registry is also load-bearing for proposal review (`teaching/review.py`) +and capability ledger evidence rows. Treating it as a free-form list will +produce drift; treating it as schema-bearing data lets the validator +refuse malformed entries before they reach the runtime. + +--- + +## Decision + +Introduce **Reviewer Registry v1** as a structured, validator-checked +schema for `docs/reviewers.yaml`. The schema is small on purpose. It must +be the minimum that satisfies ADR-0091 predicates without inviting +identity inflation. + +### Schema + +```yaml +schema_version: 1 +reviewers: + - reviewer_id: shay-j + display_name: "Joshua Shay" + role: primary + domains: ["*"] + review_scope: ["pack", "proposal", "chain", "eval"] + provenance: "adr-0092:bootstrap:2026-05-21" +``` + +| Field | Required | Purpose | +| --- | --- | --- | +| `schema_version` | yes | Version gate for registry parsing semantics. | +| `reviewer_id` | yes | Stable key referenced by pack manifests and proposals. | +| `display_name` | yes | Human-readable label for ledger and report rows. | +| `role` | yes | `primary` or `domain`. `primary` may review any pack; `domain` requires `domains` enumeration. | +| `domains` | yes | List of `domain_id` values from ADR-0091. `["*"]` for primary reviewers. | +| `review_scope` | yes | Subset of `{pack, proposal, chain, eval}`. Bounds what artifacts this reviewer may ratify. | +| `provenance` | yes | Review trail entry for the reviewer's addition itself. | + +### Bootstrap entry + +Registry ships with exactly one reviewer (`shay-j`) at v1 landing. Adding +reviewers is a separate reviewed proposal flow. The bootstrap entry is +self-sealed: its `provenance` references this ADR. + +### Validator rules + +The capability validator (ADR-0091, follow-up #4) refuses ratification +when: + +1. `docs/reviewers.yaml` does not match `schema_version: 1`. +2. A pack manifest names a `reviewer_id` absent from the registry. +3. A reviewer's `role: domain` does not include the pack's `domain_id`. +4. A reviewer's `review_scope` does not cover the artifact being ratified. +5. Two reviewer entries share a `reviewer_id`. + +No validator mutates the registry. Mutation remains a reviewed proposal. + +--- + +## Trust Boundary + +`docs/reviewers.yaml` is parsed at validator startup and on every ledger +report run. The parser rejects unknown top-level keys and unknown +reviewer fields rather than ignoring them — schema drift is loud, not +silent. Reviewer IDs are display-only; the registry never grants runtime +permissions beyond the predicates ADR-0091 already enforces. + +--- + +## Invariant + +`reviewer_registry_schema_v1` — running `core capability ledger` against +a registry that fails schema validation must exit non-zero and produce a +typed error naming the failing field; no ledger row may be emitted under +a malformed registry. + +--- + +## Lane + +`evals/reviewer_registry/` (new): + +- positive: valid v1 registry passes +- negative: empty registry blocks all `reasoning-capable` claims +- negative: malformed entry produces typed error +- negative: domain reviewer claiming wildcard `["*"]` rejected + +--- + +## Consequences + +- One real reviewer entered; subsequent reviewer additions are themselves + reviewed proposals, preventing trust inflation. +- ADR-0091 predicate #8 (reviewer resolution) becomes enforceable. +- Identity, safety, ethics packs gain a checked author trail without + altering their existing self-seal flow. + +--- + +## PR Checklist + +- Capability added: schema-checked reviewer registry; unblocks reasoning-capable claims. +- Invariant proved: `reviewer_registry_schema_v1`. +- Lane proving it: `evals/reviewer_registry/`. +- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none. +- Trust boundary: YAML parser rejects unknown keys; bootstrap entry self-sealed. diff --git a/docs/decisions/ADR-0093-domain-pack-contract-v1-implementation.md b/docs/decisions/ADR-0093-domain-pack-contract-v1-implementation.md new file mode 100644 index 00000000..1101868b --- /dev/null +++ b/docs/decisions/ADR-0093-domain-pack-contract-v1-implementation.md @@ -0,0 +1,129 @@ +# ADR-0093 — Domain Pack Contract v1 Implementation + +**Status:** Proposed +**Date:** 2026-05-21 +**Author:** CORE agents + reviewers +**Depends on:** ADR-0091, ADR-0092 + +--- + +## Context + +ADR-0091 defines the Domain Pack Contract but explicitly states: + +> The validator remains proposal-only until the schema change is +> implemented. … Runtime behavior remains unchanged until a follow-up +> implementation PR wires validation. + +The capability surface (`core capability {chains, flags, ledger, +artifact, domain_contract, evidence_plan}`) already exists and the +`docs/gaps.md` registry shows extensive structural work landed. What is +missing is the wiring that turns ADR-0091's optional manifest fields +into validator predicates that actually gate status transitions. + +Without that wiring, a domain pack can claim `domain_contract_version: 1` +in its manifest and the validator will read it but not enforce it. The +ADR-0091 follow-up list (parser support, dry-run validation, chain +registry wiring, eval lane references, reviewer metadata) is the work +this ADR implements. + +--- + +## Decision + +Implement ADR-0091's five follow-up items as a single, evidence-bearing +PR. Each item is small; bundling avoids a partial state where some +predicates fire and others silently no-op. + +### Items + +1. **Manifest parser support for `domain_contract_version=1`.** + Extend `language_packs/compiler.py` (or its loader sibling) to recognize + the optional fields enumerated in ADR-0091. Unknown values for known + fields are rejected; unknown fields are rejected (loud schema). + +2. **Dry-run validation for domain contract fields.** + New command `core capability domain-contract validate --dry-run` + runs the nine predicates from ADR-0091 §"Validation Semantics" and + reports per-predicate pass/fail without mutating state. Exit code is + non-zero on any failure. + +3. **Domain-specific chain registry wiring.** + Extend `chat/teaching_grounding.py`'s `TEACHING_CORPORA` registration + to consult pack manifest's `teaching_chains` field at mount time. + The existing first-match-wins resolution (ADR-0064) is preserved; + this only widens the registration source. + +4. **Eval lane references in capability artifact metadata.** + `core capability artifact ` includes per-lane split paths + (dev/public/holdout) from the manifest's `eval_lanes` field, plus the + most recent report SHA for each. Missing or stale reports surface as + blocked status, not silent omission. + +5. **Reviewer metadata resolution.** + `core capability domain-contract validate` consults the registry from + ADR-0092 and refuses ratification on any unresolved or + out-of-scope reviewer. + +### What is deliberately not in scope + +- No new capability status tier beyond ADR-0091's five. +- No automatic mutation of pack manifests. Manifests are still + hand-authored and reviewed. +- No retrofitting of linguistic packs that don't claim domain status. + ADR-0091's optionality is preserved. + +--- + +## Invariant + +`domain_contract_v1_predicates_enforced` — for every pack with +`domain_contract_version: 1`, running `core capability domain-contract +validate ` either passes all nine ADR-0091 predicates or emits +a typed error naming each failing predicate. No pack with one or more +failing predicates may produce a `reasoning-capable` or `expert-demo` +ledger row. + +--- + +## Lane + +`evals/domain_contract_validation/` (new): + +- positive: pack with all predicates satisfied → validator pass + ledger row eligible +- negative (per predicate): nine cases, one per predicate from ADR-0091 §"Validation Semantics", each minimally broken to confirm the validator catches it +- replay: validator output is deterministic across runs +- coincidence: pack without `domain_contract_version` field still passes ordinary `core pack validate` unchanged + +--- + +## Trust Boundary + +The validator reads pack manifests, the reviewer registry, the gaps +registry, and eval report files. All paths are sanitized via +`core/_safe_display.safe_pack_id` and existing pack-validation traversal +rejection (ADR-0051). No dynamic imports. No filesystem writes outside +deterministic report emission. + +--- + +## Consequences + +- ADR-0091 advances from `Proposed` to `Accepted` once the implementation + PR lands and this ADR's lane passes. +- The first pack ratification under ADR-0091 (ADR-0097) becomes + mechanically possible. Until then, it is blocked by missing + enforcement, not by missing content. +- Existing linguistic packs (cognition, relations, register, identity, + safety, ethics) are unchanged. Their lack of `domain_contract_version` + remains valid. + +--- + +## PR Checklist + +- Capability added: ADR-0091 enforcement wired into validator and ledger. +- Invariant proved: `domain_contract_v1_predicates_enforced`. +- Lane proving it: `evals/domain_contract_validation/`. +- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none. +- Trust boundary: pack path traversal rejection preserved; reviewer registry consulted, never mutated. diff --git a/docs/decisions/ADR-0094-proposal-source-provenance.md b/docs/decisions/ADR-0094-proposal-source-provenance.md new file mode 100644 index 00000000..0de55f65 --- /dev/null +++ b/docs/decisions/ADR-0094-proposal-source-provenance.md @@ -0,0 +1,125 @@ +# ADR-0094 — Proposal Source Provenance + +**Status:** Proposed +**Date:** 2026-05-21 +**Author:** CORE agents + reviewers + +--- + +## Context + +`teaching/proposals/proposals.jsonl` and the surrounding review flow +(`teaching/review.py`, `teaching/store.py`, ADR-0057) currently assume +proposals originate from a single source: an operator authoring through +the existing teaching CLI. The schema has no typed source field. + +Two near-term ADRs widen this: + +- ADR-0095 introduces miner-sourced proposals (contemplation Phase 5 + loop closure). +- A future ADR will introduce curriculum-sourced proposals (deferred + per the rewrite blueprint). + +Without a typed source field, downstream consumers (`teaching/review.py`, +capability ledger evidence rows, audit telemetry) will branch on string +prefixes informally. That is the same shape of mistake ADR-0067's +explicit `subject_pack_id` / `object_pack_id` fields prevented for +cross-pack chains. + +--- + +## Decision + +Introduce a sealed `ProposalSource` type widening +`PackMutationProposal` / `TeachingProposal` schemas. The widening is +schema-only; no runtime behavior changes under this ADR. + +### Sealed type + +```python +@dataclass(frozen=True, slots=True) +class ProposalSource: + kind: Literal["operator", "miner", "curriculum"] + source_id: str # "" for kind="operator"; miner_id or course_id otherwise + emitted_at_revision: str # git SHA at emission + + def serialize(self) -> str: + # "operator", "miner:articulation_quality", "curriculum:math_logic_v1" + return self.kind if not self.source_id else f"{self.kind}:{self.source_id}" +``` + +### Schema migration + +- New field on `PackMutationProposal` and `TeachingProposal`: + `source: ProposalSource`. +- Default for existing operator-authored proposals: `ProposalSource(kind="operator", source_id="", emitted_at_revision=)`. +- Migration is one-shot at ADR landing: a deterministic rewriter walks + existing `proposals.jsonl` files, attaches the default operator + source, and rewrites in sorted order. Migration is a reviewed + proposal itself. + +### Consumer rules + +- `teaching/review.py` performs exhaustive match on `source.kind`. Any + new kind requires a new ADR adding a branch. +- Telemetry events (`chat/telemetry.py`) carry `source.serialize()` as + a string field. Redact-by-default already covers proposal content; + source is non-sensitive and emitted plainly. +- Capability ledger evidence rows (ADR-0091) include `source` in their + provenance trail. + +### What this ADR does not do + +- Does not introduce miner-sourced proposals (ADR-0095). +- Does not introduce curriculum-sourced proposals. +- Does not change review thresholds. Source affects audit, not gate. + +--- + +## Invariant + +`proposal_source_exhaustive_match` — every code path that branches on +`proposal.source.kind` uses Python `match` with explicit cases for each +sealed-type value; the type checker refuses a non-exhaustive match. A +proposal without `source` fails parsing. + +--- + +## Lane + +`evals/proposal_source_schema/` (new, small): + +- positive: round-trip serialization for each `kind` +- negative: missing `source` field rejected at parse +- negative: unknown `kind` rejected at parse +- migration: existing `proposals.jsonl` rewritten deterministically; + two runs produce identical bytes + +--- + +## Trust Boundary + +`source_id` is a typed string but flows through user-adjacent surfaces +(telemetry, ledger reports). It is sanitized via +`core/_safe_display.safe_display` at all surface emission points, +matching the discipline established in ADR-0051. The migration rewriter +runs only when invoked explicitly; no implicit on-load mutation. + +--- + +## Consequences + +- ADR-0095 can introduce `kind="miner"` without secondary schema churn. +- A future curriculum ADR can introduce `kind="curriculum"` the same way. +- Existing operator review flow is unchanged; the new field is set to + its operator default on every existing proposal. + +--- + +## PR Checklist + +- Capability added: typed provenance schema for proposals. +- Invariant proved: `proposal_source_exhaustive_match`. +- Lane proving it: `evals/proposal_source_schema/`. +- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none. Migration is explicit and reviewed. +- Trust boundary: `source_id` sanitized at every surface; no implicit on-load rewrites. diff --git a/docs/decisions/ADR-0095-miner-sourced-teaching-proposals.md b/docs/decisions/ADR-0095-miner-sourced-teaching-proposals.md new file mode 100644 index 00000000..d4fa0745 --- /dev/null +++ b/docs/decisions/ADR-0095-miner-sourced-teaching-proposals.md @@ -0,0 +1,129 @@ +# ADR-0095 — Miner-Sourced Teaching Proposals + +**Status:** Proposed +**Date:** 2026-05-21 +**Author:** CORE agents + reviewers +**Depends on:** ADR-0094 + +--- + +## Context + +Phase 5 of the contemplation arc landed three miners under +`core/contemplation/miners/`: + +- `articulation_quality.py` (closed the articulation loop signal) +- `contradiction_detection.py` +- `frontier_compare.py` + +Each miner emits observations. None of them currently produce +`PackMutationProposal` candidates. The loop is one-way: contemplation +sees the gap, but the only way to close it is for an operator to +read the miner output and manually file a proposal. That breaks the +"learn from reviewed correction" arc CLAUDE.md names as load-bearing: + +> listen → comprehend → recall → think → articulate → learn from reviewed correction → replay deterministically + +This ADR closes the loop without weakening the reviewed-teaching +discipline. The doctrine constraint is sharp: there must be exactly +one correction path, and miner-sourced proposals must traverse it. + +--- + +## Decision + +Introduce `teaching/proposals/from_miner.py` that translates miner +observations into `PackMutationProposal` candidates with +`source = ProposalSource(kind="miner", source_id=, …)` from +ADR-0094. + +### Hard constraints + +1. **Single review path.** Miner-sourced proposals enter + `teaching/review.py` via the same entry point as operator proposals. + No parallel reviewer, no auto-acceptance, no shortened review path. +2. **Default status `speculative`.** Miner-sourced proposals are never + coherent at emission. +3. **Identity-pack defense.** Proposals touching identity-pack axes are + rejected at proposal-construction time (before review), not at + review time. This prevents the miner from ever filing an identity + override candidate, even one that would fail review. ADR-0027's + identity-override rejection rule extends upstream. +4. **Replay-equivalence pre-gate.** Before a miner-sourced proposal is + review-eligible, replay the originating turn under the proposed + mutation. If `trace_hash` changes on any non-target turn in the + lane, the proposal is rejected at construction. +5. **Deterministic emission.** Same miner observations + same head SHA + → byte-identical proposal stream. Proposal IDs are SHA-256 of + `(miner_id, observation_canonical, emitted_at_revision)`. + +### What miners do not gain + +- Direct write access to packs. +- Ability to mark proposals coherent. +- Ability to mutate identity, safety, or ethics packs at all. + +### Telemetry + +Miner-sourced proposals emit a `"type": "proposal_emitted"` event to +the existing telemetry sink (ADR-0040), carrying the redacted +`source.serialize()` string and proposal ID. Content is redacted by +default per ADR-0040. + +--- + +## Invariant + +`miner_proposal_replay_equivalence` — for every miner-sourced proposal +that reaches review eligibility, replaying the originating lane with +the proposed mutation applied yields identical `trace_hash` on every +non-target turn. The lane gate refuses proposals that violate this. + +`miner_proposal_single_review_path` — grep gate refuses any code path +that promotes a miner-sourced proposal to coherent outside +`teaching/review.py`. + +--- + +## Lane + +`evals/miner_loop_closure/` (new): + +- positive: legitimate articulation gap → proposal emitted → reviewable +- negative: identity-override attempt via miner → rejected at construction +- negative: proposal that breaks replay-equivalence → rejected at construction +- negative: malformed miner observation → typed error, no proposal +- coincidence: random/noise observation → no proposal under threshold +- determinism: same observations across two runs → identical proposal stream + +--- + +## Trust Boundary + +Miners read turn telemetry only; they do not read user-controlled text +directly. The miner-to-proposal boundary sanitizes `source_id` via +`safe_pack_id` traversal rejection. The replay-equivalence gate is a +filesystem-only operation against the existing eval runners; no network +or shell. + +--- + +## Consequences + +- Phase 5 contemplation becomes a closed loop in code, not just in + doctrine. +- The capability ledger gains a new evidence row class: "miner-sourced + proposal accepted into coherent" — measurable, replayable. +- Learning-scale claims (the deferred 10k harness) become eventually + defensible because every replayed proposal will have a real + provenance, not a synthetic one. + +--- + +## PR Checklist + +- Capability added: closed contemplation loop through reviewed teaching. +- Invariants proved: `miner_proposal_replay_equivalence`, `miner_proposal_single_review_path`. +- Lane proving it: `evals/miner_loop_closure/`. +- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none. Single review path enforced by grep gate. +- Trust boundary: miner reads telemetry, never user text; identity-pack rejection at construction. diff --git a/docs/decisions/ADR-0096-fabrication-control-eval-lane.md b/docs/decisions/ADR-0096-fabrication-control-eval-lane.md new file mode 100644 index 00000000..48d947be --- /dev/null +++ b/docs/decisions/ADR-0096-fabrication-control-eval-lane.md @@ -0,0 +1,118 @@ +# ADR-0096 — Fabrication-Control Eval Lane + +**Status:** Proposed +**Date:** 2026-05-21 +**Author:** CORE agents + reviewers + +--- + +## Context + +CORE has roughly 50 eval lanes. They measure many positives: +`compositionality/`, `cross_domain_transfer/`, `multi_step_reasoning/`, +`inference_closure/`, `forward_semantic_control/`, `walkthrough_chain/`. +They prove that valid compositions land and that constrained generation +prefers admissible endpoints. + +What no current lane measures is the **negative under composable +appearance** — a prompt that looks like it should compose, where the +substrate refuses to supply a bridge, and the system must refuse rather +than synthesize. + +This is the single most distinguishing measurement CORE can publish. A +deterministic refusal under composable-looking input is what +separates this architecture from a vibes-machine. ADR-0022's +forward-semantic-control proof has the shape; this ADR generalizes +that shape into a measurable lane spanning the four domain packs whose +chain corpora now exist (`math_logic`, `systems_software`, `physics`, +`hebrew_greek_textual_reasoning`). + +--- + +## Decision + +Introduce `evals/fabrication_control/` with three case classes and +hard-pinned acceptance thresholds. Mirrors the ADR-0022 +forward-semantic-control structure: constrained run + unconstrained +baseline + coincidence rate reported. + +### Case classes + +**Class A — phantom endpoint.** +Seed two disjoint chains: `A → B` and `C → D`. Ask "does A cause D?" or +"what does A lead to D through?" The substrate cannot bridge. Expected: +typed refusal via `chat/refusal.py`, `grounding_source = none`. + +**Class B — cross-pack non-bridge.** +Prompt a relation that *appears* compositional across two mounted +packs but lacks an `alignment.jsonl` edge or `teaching_chains` corpus +entry bridging them. Expected: refusal, no synthesis of a phantom +bridge. + +**Class C — sibling collapse.** +Prompt that conflates two distinguished lemmas under the same anchor +lens (e.g., ἐπιστήμη vs σύνεσις under `grc_logos_v1`, or חסד vs צδק +under `he_logos_v1`). Expected: refusal of the conflation; the +substrate must surface that the two are distinguished, not paper over. + +### Metrics (frozen at lane creation) + +| Metric | Definition | Threshold | +| --- | --- | --- | +| `fabrication_rate` | proportion of cases where surface asserts a relation absent from substrate | ≤ 0.01 | +| `refusal_recall` | proportion of cases that produce typed refusal or honest limitation | ≥ 0.95 | +| `coincidence_rate` | proportion of unconstrained-baseline runs that also land on refusal (reported, no target) | reported | +| `trace_evidence_present` | proportion of cases whose trace exposes `grounding_source` matching expectation | 1.00 | +| `grounding_source_matches_expected` | per-case `grounding_source ∈ {none, refusal_reason:*}` matches case spec | 1.00 | + +### Splits + +Three-set discipline per `docs/capability_roadmap.md` Rule 1: + +- `dev/`: ~30 cases, freely visible +- `public/`: ~30 cases, scored only at version cuts, no tuning +- `holdout/`: sealed, scored by clean-room runner + +### What this lane does not do + +- Does not introduce new refusal mechanisms. Reuses ADR-0036 typed + safety refusal and ADR-0048 `grounding_source = none` surface. +- Does not measure positive composition. That is `compositionality/`'s + job; this is its negative-control sibling. +- Does not require new packs. All four domain chain corpora already exist. + +--- + +## Invariant + +`fabrication_control_rate_bounded` — on the public split, +`fabrication_rate ≤ 0.01` across two consecutive runs. CI fails on +violation. Lane is the proof, and the proof is the lane. + +--- + +## Trust Boundary + +Lane reads pack data and runs runtime in eval mode. No filesystem +writes outside report emission paths. Holdout split sealed per Rule 1. + +--- + +## Consequences + +- First measured row evidencing "honest refusal" rather than "we + intend to refuse." CLAIMS.md Tier 2 gains a numeric row. +- The public showcase demo (ADR-0099) can cite this lane as the + evidence for its "honest unknown" scene. +- A regression in pack saturation that introduces phantom bridges + becomes visible as a `fabrication_rate` rise. + +--- + +## PR Checklist + +- Capability added: first negative-control fabrication measurement. +- Invariant proved: `fabrication_control_rate_bounded`. +- Lane proving it: `evals/fabrication_control/` itself. +- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none. +- Trust boundary: holdout sealed; eval mode read-only against runtime. diff --git a/docs/decisions/ADR-0097-mathematics-logic-reasoning-capable-ratification.md b/docs/decisions/ADR-0097-mathematics-logic-reasoning-capable-ratification.md new file mode 100644 index 00000000..f7eab761 --- /dev/null +++ b/docs/decisions/ADR-0097-mathematics-logic-reasoning-capable-ratification.md @@ -0,0 +1,155 @@ +# ADR-0097 — Mathematics-Logic Reasoning-Capable Ratification + +**Status:** Proposed +**Date:** 2026-05-21 +**Author:** CORE agents + reviewers +**Depends on:** ADR-0091, ADR-0092, ADR-0093, ADR-0096 + +--- + +## Context + +The mathematics/logic substrate is more complete than recent planning +documents acknowledged: + +- `language_packs/data/en_mathematics_logic_v1/` ships with manifest, + lexicon, glosses. +- `teaching/domain_chains/mathematics_logic_chains_v1.jsonl` exists. +- `evals/elementary_mathematics_ood/` exists. +- `docs/gaps.md` marks every mathematics-logic gap closed: + `gap:mathematics_logic_pack_absent`, + `gap:mathematics_logic_transitive_chains_below_threshold`, + `gap:mathematics_logic_proof_chain_chains_below_threshold`, + `gap:mathematics_logic_contradiction_chains_below_threshold`, + `gap:mathematics_logic_intent_shapes_below_threshold` — all `[x]`. + +What is missing is the formal ratification step under ADR-0091. The +pack manifest does not yet carry `domain_contract_version: 1` fields; +the validator (ADR-0093) cannot run against it yet; the reviewer +registry (ADR-0092) was empty until that ADR. + +This ADR is the first concrete domain claim under the Domain Pack +Contract. It is small on purpose: content already exists, infrastructure +already exists, and this is the wiring that turns existing artifacts +into a ratified `reasoning-capable` ledger row. + +--- + +## Decision + +Ratify `en_mathematics_logic_v1` as `reasoning-capable` under +ADR-0091 by emitting the contract fields into its manifest and passing +`core capability domain-contract validate en_mathematics_logic_v1`. + +### Manifest additions + +```jsonc +{ + "domain_contract_version": 1, + "domain_id": "mathematics_logic", + "axioms": null, + "rules": null, + "teaching_chains": ["mathematics_logic_chains_v1"], + "eval_lanes": [ + { + "lane": "elementary_mathematics_ood", + "version": "v1", + "splits": ["dev", "public", "holdout"] + }, + { + "lane": "inference_closure", + "version": "v1", + "splits": ["dev", "public", "holdout"] + }, + { + "lane": "fabrication_control", + "version": "v1", + "splits": ["dev", "public", "holdout"] + } + ], + "reviewers": ["shay-j"], + "known_gaps": [], + "provenance": "adr-0097:reviewed:2026-05-21" +} +``` + +`axioms` and `rules` stay `null` at v1. The pack proves reasoning +through chain composition (modus ponens, modus tollens, contradiction +detection, transitivity), not through declarative axioms. A future +ADR may add explicit axioms; this one does not. + +### Required existing evidence (no new content) + +- `mathematics_logic_chains_v1.jsonl` must contain ≥ 8 reviewed + chains per operator family it claims (per ADR-0091 predicate #5). + If it does not, ratification is blocked. Block is content, not + contract — close the chain corpus first, ratify second. +- ≥ 3 intent shapes present (per predicate #6). The cognition lane's + intent coverage (DEFINITION, RECALL, CAUSE, VERIFICATION, + COMPARISON, PROCEDURE, CORRECTION, NARRATIVE, EXAMPLE) already + exceeds this on the cognition pack; this ADR audits that the + math/logic chain corpus exercises at least three. +- Three-split eval lane referenced. `elementary_mathematics_ood` + exists; `inference_closure` exists; ADR-0096 introduces + `fabrication_control`. Holdout paths verified non-empty. + +### What this ADR does not do + +- Does not introduce new lemmas, chains, or evals. +- Does not promote to `expert-demo`. That requires audit-tour-equivalent + reports — separate ADR. +- Does not modify the runtime. Pack mounting behavior is unchanged. + +--- + +## Invariant + +`mathematics_logic_reasoning_capable_ledger_row` — running +`core capability ledger` emits a row for `domain_id: mathematics_logic` +with `status: reasoning-capable`, with provenance pointing at this +ADR, and refuses to advance to `expert-demo` until a future ADR +attaches the required reports. + +--- + +## Lane + +No new lane. Existing lanes carry the evidence: + +- `elementary_mathematics_ood/` (positive coverage) +- `inference_closure/` (composition) +- `fabrication_control/` (negative control, ADR-0096) +- `evals/domain_contract_validation/` (ADR-0093 confirms predicates fire) + +CLAIMS.md Tier 2 gains one row pointing at the four lanes and this +ADR. + +--- + +## Trust Boundary + +Manifest edit is a reviewed proposal. Checksum refresh per +ADR-0027/0029 discipline. No runtime mount path changes. No new +filesystem writes outside the manifest and its checksum companion. + +--- + +## Consequences + +- First `reasoning-capable` row in the capability ledger backed by an + ADR-0091-validated pack. +- Sibling ratifications (systems-software, physics, hebrew/greek) + become mechanical follow-ups using the same template. Each one is + its own ADR but each is small. +- The public showcase demo (ADR-0099) gains a real domain to draw + Scene 4 from. + +--- + +## PR Checklist + +- Capability added: first ratified `reasoning-capable` domain claim. +- Invariant proved: `mathematics_logic_reasoning_capable_ledger_row`. +- Lanes proving it: four existing lanes plus ADR-0093 contract validator. +- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none. Ratification is the reviewed proposal. +- Trust boundary: manifest edit through reviewed flow; checksums refreshed. diff --git a/docs/decisions/ADR-0098-demo-composition-contract.md b/docs/decisions/ADR-0098-demo-composition-contract.md new file mode 100644 index 00000000..c3cae950 --- /dev/null +++ b/docs/decisions/ADR-0098-demo-composition-contract.md @@ -0,0 +1,152 @@ +# ADR-0098 — Demo Composition Contract + +**Status:** Proposed +**Date:** 2026-05-21 +**Author:** CORE agents + reviewers + +--- + +## Context + +CORE has shipped a growing set of operator-facing demos: + +- `core demo audit-tour` (ADR-0042) +- `core demo anti-regression` (ADR-0055) +- `core demo learning-loop` (ADR-0056) +- `core demo register-tour` (ADR-0072) +- `core demo anchor-lens-tour` (ADR-0073d) +- `core demo orthogonality-tour` (ADR-0074) +- `core bench --suite teaching-loop` (ADR-0057) + +Each one is correct in isolation. Each one stands as its own evidence +of a specific invariant. The problem the next ADR (ADR-0099 public +showcase) faces is that there is no shared contract that lets one +demo safely embed another. + +Without a contract, the showcase has two bad options: +1. Reimplement portions of each demo inline (drift, duplication, + doctrine violation). +2. Subprocess-spawn each demo and parse stdout (fragile, breaks the + trace-hash discipline). + +The right answer is a small protocol that the existing demos retrofit +to, and that the showcase consumes. + +--- + +## Decision + +Introduce `DemoCommand` as a typed protocol. Existing demos are +retrofitted to it in the same PR (mechanical, small). Future demos +implement it from the start. + +### Protocol + +```python +class DemoCommand(Protocol): + demo_id: str # stable identifier, kebab-case + claim_contract_version: int # currently 1 + + def run(self, *, output_dir: Path, seed: int | None = None) -> DemoResult: ... + +@dataclass(frozen=True, slots=True) +class DemoResult: + demo_id: str + claims: tuple[Claim, ...] + evidence: Mapping[str, str] # claim_id -> evidence locator (path or sha) + all_claims_supported: bool + json_path: Path + trace_features: Mapping[str, str] # canonical, for showcase composition + +@dataclass(frozen=True, slots=True) +class Claim: + claim_id: str + statement: str + supported: bool + evidence_locator: str +``` + +### Rules + +1. **Deterministic JSON.** Two runs with the same inputs and seed + produce byte-identical `json_path` contents. HTML may differ + in formatting; JSON is the truth-path. +2. **No global state mutation.** A demo's `run()` may not mutate + process-global registries (runtime singletons, telemetry sinks + attached at module load, environment variables outside its own + scope). Demos that need a telemetry sink attach a local one and + detach it before returning. +3. **Declared output paths only.** A demo writes only under + `output_dir`. Path traversal rejected via `safe_pack_id`-class + sanitization. +4. **Composability is read-only.** A composing demo (the showcase) + may read another demo's `DemoResult` but never mutates it. + +### Retrofit scope + +Each shipped demo gains a thin adapter in +`core/commands/demo_.py` that conforms to `DemoCommand`. The +adapter does not change demo behavior; it wraps the existing entry +point and produces a `DemoResult`. + +### What this ADR does not do + +- Does not change demo behavior. +- Does not change demo CLI surface. `core demo audit-tour` runs the + same way; the protocol is internal. +- Does not introduce a registry. Demos remain discoverable via the + existing CLI subparser. + +--- + +## Invariant + +`demo_composition_no_side_effects` — a grep gate on the showcase's +import graph refuses any symbol that mutates runtime singletons or +attaches telemetry sinks at module load. The protocol contract is +enforced by structure, not by hope. + +`demo_json_byte_equality` — for each demo retrofitted under this ADR, +running it twice with identical inputs produces byte-identical JSON. +CI lane verifies. + +--- + +## Lane + +`evals/demo_composition/` (new): + +- positive: each retrofitted demo runs twice → identical JSON +- negative: a deliberately stateful test fixture → composition + detector rejects it +- composition: showcase reads two demo results → produces composite + claim set without mutating either + +--- + +## Trust Boundary + +Demos write only to operator-specified `output_dir`. Path traversal +rejection inherits from ADR-0051. No dynamic imports. No network. No +shell. + +--- + +## Consequences + +- ADR-0099 public showcase becomes mechanically possible without + reimplementing demo logic. +- Future demos cost less: implement the protocol once, gain + composability for free. +- The shipped demos gain a small adapter layer but no behavioral + change. + +--- + +## PR Checklist + +- Capability added: composition protocol for demos. +- Invariants proved: `demo_composition_no_side_effects`, `demo_json_byte_equality`. +- Lane proving it: `evals/demo_composition/`. +- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none. +- Trust boundary: demos write only under declared output paths; no global state mutation. diff --git a/docs/decisions/ADR-0099-public-showcase-demo.md b/docs/decisions/ADR-0099-public-showcase-demo.md new file mode 100644 index 00000000..5e388152 --- /dev/null +++ b/docs/decisions/ADR-0099-public-showcase-demo.md @@ -0,0 +1,156 @@ +# ADR-0099 — Public Showcase Demo + +**Status:** Proposed +**Date:** 2026-05-21 +**Author:** CORE agents + reviewers +**Depends on:** ADR-0096, ADR-0097, ADR-0098 + +--- + +## Context + +CORE's distinctive properties (deterministic cognition, honest +refusal, reviewed learning, multi-hop composition with replayable +trace) are each measured by a different shipped artifact. A first-time +viewer has to read seven ADRs to assemble the thesis. + +The single best forcing function is one artifact that demonstrates all +four properties in under 30 seconds, with each scene cross-linked to +the lane that proves it. The pattern was established by +`core demo audit-tour` (ADR-0042): a JSON claim contract gated by +`all_claims_supported=True`. This ADR generalizes that pattern across +the four key invariants. + +The constraint that keeps the demo honest is in ADR-0098: it must +compose existing demos, not reimplement them. Every claim the showcase +makes traces back to an already-passing lane. + +--- + +## Decision + +Introduce `core demo showcase` that composes four scenes, each +delegating to a `DemoCommand` (ADR-0098). Output is a single +deterministic JSON contract plus an HTML render. + +### Scenes + +**Scene 1 — Determinism.** +Delegate: existing `register-tour` `DemoCommand` adapter. +Claim: "Identical prompt produces identical trace hash across 10 runs +under each shipped register." +Evidence: `register-tour` JSON; `trace_hash` set cardinality = 1 per +register. + +**Scene 2 — Honest unknown.** +Delegate: thin `DemoCommand` wrapper around the `fabrication_control` +public split (ADR-0096). +Claim: "Composable-looking but unsupported prompts produce typed +refusal with `grounding_source = none`." +Evidence: `fabrication_control` report SHA; +`fabrication_rate ≤ 0.01`. + +**Scene 3 — Reviewed learning.** +Delegate: existing `learning-loop` `DemoCommand` adapter. +Claim: "Speculative teaching is marked speculative until reviewed; +after review, identical prompt produces coherent answer." +Evidence: `learning-loop` JSON; pre-review status `speculative`, +post-review status `coherent`, byte-equal surface across replay. + +**Scene 4 — Multi-hop with trace.** +Delegate: new `DemoCommand` (thin) that runs one transitive prompt +against the math/logic pack ratified in ADR-0097. +Claim: "Multi-hop reasoning produces an answer plus a verifiable +operator trace." +Evidence: turn output with `grounding_source = pack`, operator +invocation in trace, replay byte-equal. + +### Output contract + +```jsonc +{ + "showcase_version": 1, + "generated_at_revision": "", + "scenes": [ + { + "scene_id": "determinism", + "demo_id": "register-tour", + "claims": [...], + "all_claims_supported": true + }, + ... + ], + "all_claims_supported": true, + "total_runtime_ms": ..., + "trace_hash": "..." +} +``` + +### Hard constraints + +- **Total runtime <30s on dev hardware.** If a scene exceeds budget, + the scene is trimmed before the runtime constraint is relaxed. +- **No new mechanism.** Grep gate on the showcase's import graph + refuses any symbol not already exported by a shipped module. +- **JSON byte-equality across runs.** HTML may vary in styling; JSON + must not. Replay verified in CI. +- **Public-safety preserved.** Every surface emitted is already + emitted by an existing public demo. No new exposure. + +--- + +## Invariant + +`public_showcase_pure_composition` — grep gate refuses any symbol in +the showcase's import graph not already exported by `core/`, `chat/`, +`generate/`, `language_packs/`, `teaching/`, or `core.commands.demo_*`. + +`public_showcase_all_claims_supported` — CI fails if showcase exits +with `all_claims_supported=False` or runtime >30s. + +`public_showcase_json_byte_equality` — two consecutive showcase runs +produce byte-identical JSON. + +--- + +## Lane + +`evals/public_demo/` (existing directory, populated by this ADR): + +- determinism: showcase JSON byte-equal across two runs +- support: `all_claims_supported=True` +- budget: total runtime <30s on dev hardware reference machine +- composition: grep gate confirms pure composition +- scene-level: each scene's underlying demo lane also passes + +--- + +## Trust Boundary + +Showcase writes under operator-specified `--output-dir`. Path traversal +rejection per ADR-0051 / ADR-0098. No network. No shell. No dynamic +imports. HTML is generated from JSON via a static template; no +operator-supplied template path. + +--- + +## Consequences + +- One artifact answers "what makes CORE distinct" in under 30 seconds. +- Every claim in that artifact is backed by an already-passing eval + lane; no marketing layer. +- The showcase becomes the natural regression sentinel: if any of the + four underlying invariants weakens, the showcase fails before any + external audience sees it. +- ADRs 0084 through 0098 each contribute exactly one piece of evidence + to the showcase. The slate closes. + +--- + +## PR Checklist + +- Capability added: single artifact composing four CORE invariants under 30s. +- Invariants proved: `public_showcase_pure_composition`, `public_showcase_all_claims_supported`, `public_showcase_json_byte_equality`. +- Lane proving it: `evals/public_demo/`. +- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none. Pure composition enforced by grep gate. +- Trust boundary: writes only under declared output path; no operator-supplied templates.