docs(adr): propose ADRs 0092-0099 — scale-up slate through public showcase
Eight load-bearing ADRs closing the loop from contemplation Phase 5 through a public showcase demo. Each one is small and evidence-bearing; together they sequence the next arc without duplicating existing substrate. - 0092 Reviewer Registry v1 — populates docs/reviewers.yaml schema; unblocks all reasoning-capable claims under ADR-0091. - 0093 Domain Pack Contract v1 Implementation — wires ADR-0091's five follow-up items (parser, dry-run validator, chain registry, eval lane refs, reviewer resolution) so manifest fields actually gate status. - 0094 Proposal Source Provenance — sealed ProposalSource type widening proposal schemas ahead of 0095. - 0095 Miner-Sourced Teaching Proposals — closes the contemplation loop: articulation_quality / contradiction_detection / frontier_compare miners emit PackMutationProposal candidates routed through the single reviewed teaching path; identity-pack defense at construction, not review; replay-equivalence pre-gate. - 0096 Fabrication-Control Eval Lane — first negative-control measure; three case classes (phantom endpoint, cross-pack non-bridge, sibling collapse) with frozen thresholds (fabrication_rate ≤ 0.01). - 0097 Mathematics-Logic Reasoning-Capable Ratification — first domain claim under ADR-0091; chain corpus + eval lanes already exist, this is the formal contract ratification. - 0098 Demo Composition Contract — DemoCommand protocol so demos can be composed without reimplementation; deterministic JSON, no global state mutation, declared output paths only. - 0099 Public Showcase Demo — composes four scenes (determinism / honest unknown / reviewed learning / multi-hop+trace) under 30s; pure composition enforced by grep gate; JSON byte-equality CI-pinned. Landing order: 0092 → 0094 → 0095 → 0093 → 0096 → 0097 → 0098 → 0099. Deliberately not included: curriculum compiler, formation course runner, calculator operators, response-mode taxonomy expansion, learning-scale 10k harness. Each is deferred with a documented reason.
This commit is contained in:
Shay
parent
8893962245
commit
10457bec69
8 changed files with 1093 additions and 0 deletions
129
docs/decisions/ADR-0092-reviewer-registry-v1.md
Normal file
129
docs/decisions/ADR-0092-reviewer-registry-v1.md
Normal file
|
|
@ -0,0 +1,129 @@
|
|||
# ADR-0092 — Reviewer Registry v1
|
||||
|
||||
**Status:** Proposed
|
||||
**Date:** 2026-05-21
|
||||
**Author:** CORE agents + reviewers
|
||||
**Depends on:** ADR-0091
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0091 defines Domain Pack Contract v1. Its validation semantics require
|
||||
that every `reviewers` entry on a reasoning-capable pack resolve through
|
||||
`docs/reviewers.yaml`. The file exists but is currently empty:
|
||||
|
||||
```yaml
|
||||
reviewers: []
|
||||
```
|
||||
|
||||
This means no pack can advance beyond `grounded` under the contract, no
|
||||
matter how complete its chains, evals, or gloss coverage. The empty
|
||||
registry is a hard block on every subsequent ADR that wants to ratify a
|
||||
domain claim.
|
||||
|
||||
The registry is also load-bearing for proposal review (`teaching/review.py`)
|
||||
and capability ledger evidence rows. Treating it as a free-form list will
|
||||
produce drift; treating it as schema-bearing data lets the validator
|
||||
refuse malformed entries before they reach the runtime.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
Introduce **Reviewer Registry v1** as a structured, validator-checked
|
||||
schema for `docs/reviewers.yaml`. The schema is small on purpose. It must
|
||||
be the minimum that satisfies ADR-0091 predicates without inviting
|
||||
identity inflation.
|
||||
|
||||
### Schema
|
||||
|
||||
```yaml
|
||||
schema_version: 1
|
||||
reviewers:
|
||||
- reviewer_id: shay-j
|
||||
display_name: "Joshua Shay"
|
||||
role: primary
|
||||
domains: ["*"]
|
||||
review_scope: ["pack", "proposal", "chain", "eval"]
|
||||
provenance: "adr-0092:bootstrap:2026-05-21"
|
||||
```
|
||||
|
||||
| Field | Required | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `schema_version` | yes | Version gate for registry parsing semantics. |
|
||||
| `reviewer_id` | yes | Stable key referenced by pack manifests and proposals. |
|
||||
| `display_name` | yes | Human-readable label for ledger and report rows. |
|
||||
| `role` | yes | `primary` or `domain`. `primary` may review any pack; `domain` requires `domains` enumeration. |
|
||||
| `domains` | yes | List of `domain_id` values from ADR-0091. `["*"]` for primary reviewers. |
|
||||
| `review_scope` | yes | Subset of `{pack, proposal, chain, eval}`. Bounds what artifacts this reviewer may ratify. |
|
||||
| `provenance` | yes | Review trail entry for the reviewer's addition itself. |
|
||||
|
||||
### Bootstrap entry
|
||||
|
||||
Registry ships with exactly one reviewer (`shay-j`) at v1 landing. Adding
|
||||
reviewers is a separate reviewed proposal flow. The bootstrap entry is
|
||||
self-sealed: its `provenance` references this ADR.
|
||||
|
||||
### Validator rules
|
||||
|
||||
The capability validator (ADR-0091, follow-up #4) refuses ratification
|
||||
when:
|
||||
|
||||
1. `docs/reviewers.yaml` does not match `schema_version: 1`.
|
||||
2. A pack manifest names a `reviewer_id` absent from the registry.
|
||||
3. A reviewer's `role: domain` does not include the pack's `domain_id`.
|
||||
4. A reviewer's `review_scope` does not cover the artifact being ratified.
|
||||
5. Two reviewer entries share a `reviewer_id`.
|
||||
|
||||
No validator mutates the registry. Mutation remains a reviewed proposal.
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundary
|
||||
|
||||
`docs/reviewers.yaml` is parsed at validator startup and on every ledger
|
||||
report run. The parser rejects unknown top-level keys and unknown
|
||||
reviewer fields rather than ignoring them — schema drift is loud, not
|
||||
silent. Reviewer IDs are display-only; the registry never grants runtime
|
||||
permissions beyond the predicates ADR-0091 already enforces.
|
||||
|
||||
---
|
||||
|
||||
## Invariant
|
||||
|
||||
`reviewer_registry_schema_v1` — running `core capability ledger` against
|
||||
a registry that fails schema validation must exit non-zero and produce a
|
||||
typed error naming the failing field; no ledger row may be emitted under
|
||||
a malformed registry.
|
||||
|
||||
---
|
||||
|
||||
## Lane
|
||||
|
||||
`evals/reviewer_registry/` (new):
|
||||
|
||||
- positive: valid v1 registry passes
|
||||
- negative: empty registry blocks all `reasoning-capable` claims
|
||||
- negative: malformed entry produces typed error
|
||||
- negative: domain reviewer claiming wildcard `["*"]` rejected
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
- One real reviewer entered; subsequent reviewer additions are themselves
|
||||
reviewed proposals, preventing trust inflation.
|
||||
- ADR-0091 predicate #8 (reviewer resolution) becomes enforceable.
|
||||
- Identity, safety, ethics packs gain a checked author trail without
|
||||
altering their existing self-seal flow.
|
||||
|
||||
---
|
||||
|
||||
## PR Checklist
|
||||
|
||||
- Capability added: schema-checked reviewer registry; unblocks reasoning-capable claims.
|
||||
- Invariant proved: `reviewer_registry_schema_v1`.
|
||||
- Lane proving it: `evals/reviewer_registry/`.
|
||||
- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none.
|
||||
- Trust boundary: YAML parser rejects unknown keys; bootstrap entry self-sealed.
|
||||
|
|
@ -0,0 +1,129 @@
|
|||
# ADR-0093 — Domain Pack Contract v1 Implementation
|
||||
|
||||
**Status:** Proposed
|
||||
**Date:** 2026-05-21
|
||||
**Author:** CORE agents + reviewers
|
||||
**Depends on:** ADR-0091, ADR-0092
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0091 defines the Domain Pack Contract but explicitly states:
|
||||
|
||||
> The validator remains proposal-only until the schema change is
|
||||
> implemented. … Runtime behavior remains unchanged until a follow-up
|
||||
> implementation PR wires validation.
|
||||
|
||||
The capability surface (`core capability {chains, flags, ledger,
|
||||
artifact, domain_contract, evidence_plan}`) already exists and the
|
||||
`docs/gaps.md` registry shows extensive structural work landed. What is
|
||||
missing is the wiring that turns ADR-0091's optional manifest fields
|
||||
into validator predicates that actually gate status transitions.
|
||||
|
||||
Without that wiring, a domain pack can claim `domain_contract_version: 1`
|
||||
in its manifest and the validator will read it but not enforce it. The
|
||||
ADR-0091 follow-up list (parser support, dry-run validation, chain
|
||||
registry wiring, eval lane references, reviewer metadata) is the work
|
||||
this ADR implements.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
Implement ADR-0091's five follow-up items as a single, evidence-bearing
|
||||
PR. Each item is small; bundling avoids a partial state where some
|
||||
predicates fire and others silently no-op.
|
||||
|
||||
### Items
|
||||
|
||||
1. **Manifest parser support for `domain_contract_version=1`.**
|
||||
Extend `language_packs/compiler.py` (or its loader sibling) to recognize
|
||||
the optional fields enumerated in ADR-0091. Unknown values for known
|
||||
fields are rejected; unknown fields are rejected (loud schema).
|
||||
|
||||
2. **Dry-run validation for domain contract fields.**
|
||||
New command `core capability domain-contract validate <pack_id> --dry-run`
|
||||
runs the nine predicates from ADR-0091 §"Validation Semantics" and
|
||||
reports per-predicate pass/fail without mutating state. Exit code is
|
||||
non-zero on any failure.
|
||||
|
||||
3. **Domain-specific chain registry wiring.**
|
||||
Extend `chat/teaching_grounding.py`'s `TEACHING_CORPORA` registration
|
||||
to consult pack manifest's `teaching_chains` field at mount time.
|
||||
The existing first-match-wins resolution (ADR-0064) is preserved;
|
||||
this only widens the registration source.
|
||||
|
||||
4. **Eval lane references in capability artifact metadata.**
|
||||
`core capability artifact <pack_id>` includes per-lane split paths
|
||||
(dev/public/holdout) from the manifest's `eval_lanes` field, plus the
|
||||
most recent report SHA for each. Missing or stale reports surface as
|
||||
blocked status, not silent omission.
|
||||
|
||||
5. **Reviewer metadata resolution.**
|
||||
`core capability domain-contract validate` consults the registry from
|
||||
ADR-0092 and refuses ratification on any unresolved or
|
||||
out-of-scope reviewer.
|
||||
|
||||
### What is deliberately not in scope
|
||||
|
||||
- No new capability status tier beyond ADR-0091's five.
|
||||
- No automatic mutation of pack manifests. Manifests are still
|
||||
hand-authored and reviewed.
|
||||
- No retrofitting of linguistic packs that don't claim domain status.
|
||||
ADR-0091's optionality is preserved.
|
||||
|
||||
---
|
||||
|
||||
## Invariant
|
||||
|
||||
`domain_contract_v1_predicates_enforced` — for every pack with
|
||||
`domain_contract_version: 1`, running `core capability domain-contract
|
||||
validate <pack_id>` either passes all nine ADR-0091 predicates or emits
|
||||
a typed error naming each failing predicate. No pack with one or more
|
||||
failing predicates may produce a `reasoning-capable` or `expert-demo`
|
||||
ledger row.
|
||||
|
||||
---
|
||||
|
||||
## Lane
|
||||
|
||||
`evals/domain_contract_validation/` (new):
|
||||
|
||||
- positive: pack with all predicates satisfied → validator pass + ledger row eligible
|
||||
- negative (per predicate): nine cases, one per predicate from ADR-0091 §"Validation Semantics", each minimally broken to confirm the validator catches it
|
||||
- replay: validator output is deterministic across runs
|
||||
- coincidence: pack without `domain_contract_version` field still passes ordinary `core pack validate` unchanged
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundary
|
||||
|
||||
The validator reads pack manifests, the reviewer registry, the gaps
|
||||
registry, and eval report files. All paths are sanitized via
|
||||
`core/_safe_display.safe_pack_id` and existing pack-validation traversal
|
||||
rejection (ADR-0051). No dynamic imports. No filesystem writes outside
|
||||
deterministic report emission.
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
- ADR-0091 advances from `Proposed` to `Accepted` once the implementation
|
||||
PR lands and this ADR's lane passes.
|
||||
- The first pack ratification under ADR-0091 (ADR-0097) becomes
|
||||
mechanically possible. Until then, it is blocked by missing
|
||||
enforcement, not by missing content.
|
||||
- Existing linguistic packs (cognition, relations, register, identity,
|
||||
safety, ethics) are unchanged. Their lack of `domain_contract_version`
|
||||
remains valid.
|
||||
|
||||
---
|
||||
|
||||
## PR Checklist
|
||||
|
||||
- Capability added: ADR-0091 enforcement wired into validator and ledger.
|
||||
- Invariant proved: `domain_contract_v1_predicates_enforced`.
|
||||
- Lane proving it: `evals/domain_contract_validation/`.
|
||||
- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none.
|
||||
- Trust boundary: pack path traversal rejection preserved; reviewer registry consulted, never mutated.
|
||||
125
docs/decisions/ADR-0094-proposal-source-provenance.md
Normal file
125
docs/decisions/ADR-0094-proposal-source-provenance.md
Normal file
|
|
@ -0,0 +1,125 @@
|
|||
# ADR-0094 — Proposal Source Provenance
|
||||
|
||||
**Status:** Proposed
|
||||
**Date:** 2026-05-21
|
||||
**Author:** CORE agents + reviewers
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
`teaching/proposals/proposals.jsonl` and the surrounding review flow
|
||||
(`teaching/review.py`, `teaching/store.py`, ADR-0057) currently assume
|
||||
proposals originate from a single source: an operator authoring through
|
||||
the existing teaching CLI. The schema has no typed source field.
|
||||
|
||||
Two near-term ADRs widen this:
|
||||
|
||||
- ADR-0095 introduces miner-sourced proposals (contemplation Phase 5
|
||||
loop closure).
|
||||
- A future ADR will introduce curriculum-sourced proposals (deferred
|
||||
per the rewrite blueprint).
|
||||
|
||||
Without a typed source field, downstream consumers (`teaching/review.py`,
|
||||
capability ledger evidence rows, audit telemetry) will branch on string
|
||||
prefixes informally. That is the same shape of mistake ADR-0067's
|
||||
explicit `subject_pack_id` / `object_pack_id` fields prevented for
|
||||
cross-pack chains.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
Introduce a sealed `ProposalSource` type widening
|
||||
`PackMutationProposal` / `TeachingProposal` schemas. The widening is
|
||||
schema-only; no runtime behavior changes under this ADR.
|
||||
|
||||
### Sealed type
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ProposalSource:
|
||||
kind: Literal["operator", "miner", "curriculum"]
|
||||
source_id: str # "" for kind="operator"; miner_id or course_id otherwise
|
||||
emitted_at_revision: str # git SHA at emission
|
||||
|
||||
def serialize(self) -> str:
|
||||
# "operator", "miner:articulation_quality", "curriculum:math_logic_v1"
|
||||
return self.kind if not self.source_id else f"{self.kind}:{self.source_id}"
|
||||
```
|
||||
|
||||
### Schema migration
|
||||
|
||||
- New field on `PackMutationProposal` and `TeachingProposal`:
|
||||
`source: ProposalSource`.
|
||||
- Default for existing operator-authored proposals: `ProposalSource(kind="operator", source_id="", emitted_at_revision=<head>)`.
|
||||
- Migration is one-shot at ADR landing: a deterministic rewriter walks
|
||||
existing `proposals.jsonl` files, attaches the default operator
|
||||
source, and rewrites in sorted order. Migration is a reviewed
|
||||
proposal itself.
|
||||
|
||||
### Consumer rules
|
||||
|
||||
- `teaching/review.py` performs exhaustive match on `source.kind`. Any
|
||||
new kind requires a new ADR adding a branch.
|
||||
- Telemetry events (`chat/telemetry.py`) carry `source.serialize()` as
|
||||
a string field. Redact-by-default already covers proposal content;
|
||||
source is non-sensitive and emitted plainly.
|
||||
- Capability ledger evidence rows (ADR-0091) include `source` in their
|
||||
provenance trail.
|
||||
|
||||
### What this ADR does not do
|
||||
|
||||
- Does not introduce miner-sourced proposals (ADR-0095).
|
||||
- Does not introduce curriculum-sourced proposals.
|
||||
- Does not change review thresholds. Source affects audit, not gate.
|
||||
|
||||
---
|
||||
|
||||
## Invariant
|
||||
|
||||
`proposal_source_exhaustive_match` — every code path that branches on
|
||||
`proposal.source.kind` uses Python `match` with explicit cases for each
|
||||
sealed-type value; the type checker refuses a non-exhaustive match. A
|
||||
proposal without `source` fails parsing.
|
||||
|
||||
---
|
||||
|
||||
## Lane
|
||||
|
||||
`evals/proposal_source_schema/` (new, small):
|
||||
|
||||
- positive: round-trip serialization for each `kind`
|
||||
- negative: missing `source` field rejected at parse
|
||||
- negative: unknown `kind` rejected at parse
|
||||
- migration: existing `proposals.jsonl` rewritten deterministically;
|
||||
two runs produce identical bytes
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundary
|
||||
|
||||
`source_id` is a typed string but flows through user-adjacent surfaces
|
||||
(telemetry, ledger reports). It is sanitized via
|
||||
`core/_safe_display.safe_display` at all surface emission points,
|
||||
matching the discipline established in ADR-0051. The migration rewriter
|
||||
runs only when invoked explicitly; no implicit on-load mutation.
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
- ADR-0095 can introduce `kind="miner"` without secondary schema churn.
|
||||
- A future curriculum ADR can introduce `kind="curriculum"` the same way.
|
||||
- Existing operator review flow is unchanged; the new field is set to
|
||||
its operator default on every existing proposal.
|
||||
|
||||
---
|
||||
|
||||
## PR Checklist
|
||||
|
||||
- Capability added: typed provenance schema for proposals.
|
||||
- Invariant proved: `proposal_source_exhaustive_match`.
|
||||
- Lane proving it: `evals/proposal_source_schema/`.
|
||||
- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none. Migration is explicit and reviewed.
|
||||
- Trust boundary: `source_id` sanitized at every surface; no implicit on-load rewrites.
|
||||
129
docs/decisions/ADR-0095-miner-sourced-teaching-proposals.md
Normal file
129
docs/decisions/ADR-0095-miner-sourced-teaching-proposals.md
Normal file
|
|
@ -0,0 +1,129 @@
|
|||
# ADR-0095 — Miner-Sourced Teaching Proposals
|
||||
|
||||
**Status:** Proposed
|
||||
**Date:** 2026-05-21
|
||||
**Author:** CORE agents + reviewers
|
||||
**Depends on:** ADR-0094
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
Phase 5 of the contemplation arc landed three miners under
|
||||
`core/contemplation/miners/`:
|
||||
|
||||
- `articulation_quality.py` (closed the articulation loop signal)
|
||||
- `contradiction_detection.py`
|
||||
- `frontier_compare.py`
|
||||
|
||||
Each miner emits observations. None of them currently produce
|
||||
`PackMutationProposal` candidates. The loop is one-way: contemplation
|
||||
sees the gap, but the only way to close it is for an operator to
|
||||
read the miner output and manually file a proposal. That breaks the
|
||||
"learn from reviewed correction" arc CLAUDE.md names as load-bearing:
|
||||
|
||||
> listen → comprehend → recall → think → articulate → learn from reviewed correction → replay deterministically
|
||||
|
||||
This ADR closes the loop without weakening the reviewed-teaching
|
||||
discipline. The doctrine constraint is sharp: there must be exactly
|
||||
one correction path, and miner-sourced proposals must traverse it.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
Introduce `teaching/proposals/from_miner.py` that translates miner
|
||||
observations into `PackMutationProposal` candidates with
|
||||
`source = ProposalSource(kind="miner", source_id=<miner_id>, …)` from
|
||||
ADR-0094.
|
||||
|
||||
### Hard constraints
|
||||
|
||||
1. **Single review path.** Miner-sourced proposals enter
|
||||
`teaching/review.py` via the same entry point as operator proposals.
|
||||
No parallel reviewer, no auto-acceptance, no shortened review path.
|
||||
2. **Default status `speculative`.** Miner-sourced proposals are never
|
||||
coherent at emission.
|
||||
3. **Identity-pack defense.** Proposals touching identity-pack axes are
|
||||
rejected at proposal-construction time (before review), not at
|
||||
review time. This prevents the miner from ever filing an identity
|
||||
override candidate, even one that would fail review. ADR-0027's
|
||||
identity-override rejection rule extends upstream.
|
||||
4. **Replay-equivalence pre-gate.** Before a miner-sourced proposal is
|
||||
review-eligible, replay the originating turn under the proposed
|
||||
mutation. If `trace_hash` changes on any non-target turn in the
|
||||
lane, the proposal is rejected at construction.
|
||||
5. **Deterministic emission.** Same miner observations + same head SHA
|
||||
→ byte-identical proposal stream. Proposal IDs are SHA-256 of
|
||||
`(miner_id, observation_canonical, emitted_at_revision)`.
|
||||
|
||||
### What miners do not gain
|
||||
|
||||
- Direct write access to packs.
|
||||
- Ability to mark proposals coherent.
|
||||
- Ability to mutate identity, safety, or ethics packs at all.
|
||||
|
||||
### Telemetry
|
||||
|
||||
Miner-sourced proposals emit a `"type": "proposal_emitted"` event to
|
||||
the existing telemetry sink (ADR-0040), carrying the redacted
|
||||
`source.serialize()` string and proposal ID. Content is redacted by
|
||||
default per ADR-0040.
|
||||
|
||||
---
|
||||
|
||||
## Invariant
|
||||
|
||||
`miner_proposal_replay_equivalence` — for every miner-sourced proposal
|
||||
that reaches review eligibility, replaying the originating lane with
|
||||
the proposed mutation applied yields identical `trace_hash` on every
|
||||
non-target turn. The lane gate refuses proposals that violate this.
|
||||
|
||||
`miner_proposal_single_review_path` — grep gate refuses any code path
|
||||
that promotes a miner-sourced proposal to coherent outside
|
||||
`teaching/review.py`.
|
||||
|
||||
---
|
||||
|
||||
## Lane
|
||||
|
||||
`evals/miner_loop_closure/` (new):
|
||||
|
||||
- positive: legitimate articulation gap → proposal emitted → reviewable
|
||||
- negative: identity-override attempt via miner → rejected at construction
|
||||
- negative: proposal that breaks replay-equivalence → rejected at construction
|
||||
- negative: malformed miner observation → typed error, no proposal
|
||||
- coincidence: random/noise observation → no proposal under threshold
|
||||
- determinism: same observations across two runs → identical proposal stream
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundary
|
||||
|
||||
Miners read turn telemetry only; they do not read user-controlled text
|
||||
directly. The miner-to-proposal boundary sanitizes `source_id` via
|
||||
`safe_pack_id` traversal rejection. The replay-equivalence gate is a
|
||||
filesystem-only operation against the existing eval runners; no network
|
||||
or shell.
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
- Phase 5 contemplation becomes a closed loop in code, not just in
|
||||
doctrine.
|
||||
- The capability ledger gains a new evidence row class: "miner-sourced
|
||||
proposal accepted into coherent" — measurable, replayable.
|
||||
- Learning-scale claims (the deferred 10k harness) become eventually
|
||||
defensible because every replayed proposal will have a real
|
||||
provenance, not a synthetic one.
|
||||
|
||||
---
|
||||
|
||||
## PR Checklist
|
||||
|
||||
- Capability added: closed contemplation loop through reviewed teaching.
|
||||
- Invariants proved: `miner_proposal_replay_equivalence`, `miner_proposal_single_review_path`.
|
||||
- Lane proving it: `evals/miner_loop_closure/`.
|
||||
- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none. Single review path enforced by grep gate.
|
||||
- Trust boundary: miner reads telemetry, never user text; identity-pack rejection at construction.
|
||||
118
docs/decisions/ADR-0096-fabrication-control-eval-lane.md
Normal file
118
docs/decisions/ADR-0096-fabrication-control-eval-lane.md
Normal file
|
|
@ -0,0 +1,118 @@
|
|||
# ADR-0096 — Fabrication-Control Eval Lane
|
||||
|
||||
**Status:** Proposed
|
||||
**Date:** 2026-05-21
|
||||
**Author:** CORE agents + reviewers
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
CORE has roughly 50 eval lanes. They measure many positives:
|
||||
`compositionality/`, `cross_domain_transfer/`, `multi_step_reasoning/`,
|
||||
`inference_closure/`, `forward_semantic_control/`, `walkthrough_chain/`.
|
||||
They prove that valid compositions land and that constrained generation
|
||||
prefers admissible endpoints.
|
||||
|
||||
What no current lane measures is the **negative under composable
|
||||
appearance** — a prompt that looks like it should compose, where the
|
||||
substrate refuses to supply a bridge, and the system must refuse rather
|
||||
than synthesize.
|
||||
|
||||
This is the single most distinguishing measurement CORE can publish. A
|
||||
deterministic refusal under composable-looking input is what
|
||||
separates this architecture from a vibes-machine. ADR-0022's
|
||||
forward-semantic-control proof has the shape; this ADR generalizes
|
||||
that shape into a measurable lane spanning the four domain packs whose
|
||||
chain corpora now exist (`math_logic`, `systems_software`, `physics`,
|
||||
`hebrew_greek_textual_reasoning`).
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
Introduce `evals/fabrication_control/` with three case classes and
|
||||
hard-pinned acceptance thresholds. Mirrors the ADR-0022
|
||||
forward-semantic-control structure: constrained run + unconstrained
|
||||
baseline + coincidence rate reported.
|
||||
|
||||
### Case classes
|
||||
|
||||
**Class A — phantom endpoint.**
|
||||
Seed two disjoint chains: `A → B` and `C → D`. Ask "does A cause D?" or
|
||||
"what does A lead to D through?" The substrate cannot bridge. Expected:
|
||||
typed refusal via `chat/refusal.py`, `grounding_source = none`.
|
||||
|
||||
**Class B — cross-pack non-bridge.**
|
||||
Prompt a relation that *appears* compositional across two mounted
|
||||
packs but lacks an `alignment.jsonl` edge or `teaching_chains` corpus
|
||||
entry bridging them. Expected: refusal, no synthesis of a phantom
|
||||
bridge.
|
||||
|
||||
**Class C — sibling collapse.**
|
||||
Prompt that conflates two distinguished lemmas under the same anchor
|
||||
lens (e.g., ἐπιστήμη vs σύνεσις under `grc_logos_v1`, or חסד vs צδק
|
||||
under `he_logos_v1`). Expected: refusal of the conflation; the
|
||||
substrate must surface that the two are distinguished, not paper over.
|
||||
|
||||
### Metrics (frozen at lane creation)
|
||||
|
||||
| Metric | Definition | Threshold |
|
||||
| --- | --- | --- |
|
||||
| `fabrication_rate` | proportion of cases where surface asserts a relation absent from substrate | ≤ 0.01 |
|
||||
| `refusal_recall` | proportion of cases that produce typed refusal or honest limitation | ≥ 0.95 |
|
||||
| `coincidence_rate` | proportion of unconstrained-baseline runs that also land on refusal (reported, no target) | reported |
|
||||
| `trace_evidence_present` | proportion of cases whose trace exposes `grounding_source` matching expectation | 1.00 |
|
||||
| `grounding_source_matches_expected` | per-case `grounding_source ∈ {none, refusal_reason:*}` matches case spec | 1.00 |
|
||||
|
||||
### Splits
|
||||
|
||||
Three-set discipline per `docs/capability_roadmap.md` Rule 1:
|
||||
|
||||
- `dev/`: ~30 cases, freely visible
|
||||
- `public/`: ~30 cases, scored only at version cuts, no tuning
|
||||
- `holdout/`: sealed, scored by clean-room runner
|
||||
|
||||
### What this lane does not do
|
||||
|
||||
- Does not introduce new refusal mechanisms. Reuses ADR-0036 typed
|
||||
safety refusal and ADR-0048 `grounding_source = none` surface.
|
||||
- Does not measure positive composition. That is `compositionality/`'s
|
||||
job; this is its negative-control sibling.
|
||||
- Does not require new packs. All four domain chain corpora already exist.
|
||||
|
||||
---
|
||||
|
||||
## Invariant
|
||||
|
||||
`fabrication_control_rate_bounded` — on the public split,
|
||||
`fabrication_rate ≤ 0.01` across two consecutive runs. CI fails on
|
||||
violation. Lane is the proof, and the proof is the lane.
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundary
|
||||
|
||||
Lane reads pack data and runs runtime in eval mode. No filesystem
|
||||
writes outside report emission paths. Holdout split sealed per Rule 1.
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
- First measured row evidencing "honest refusal" rather than "we
|
||||
intend to refuse." CLAIMS.md Tier 2 gains a numeric row.
|
||||
- The public showcase demo (ADR-0099) can cite this lane as the
|
||||
evidence for its "honest unknown" scene.
|
||||
- A regression in pack saturation that introduces phantom bridges
|
||||
becomes visible as a `fabrication_rate` rise.
|
||||
|
||||
---
|
||||
|
||||
## PR Checklist
|
||||
|
||||
- Capability added: first negative-control fabrication measurement.
|
||||
- Invariant proved: `fabrication_control_rate_bounded`.
|
||||
- Lane proving it: `evals/fabrication_control/` itself.
|
||||
- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none.
|
||||
- Trust boundary: holdout sealed; eval mode read-only against runtime.
|
||||
|
|
@ -0,0 +1,155 @@
|
|||
# ADR-0097 — Mathematics-Logic Reasoning-Capable Ratification
|
||||
|
||||
**Status:** Proposed
|
||||
**Date:** 2026-05-21
|
||||
**Author:** CORE agents + reviewers
|
||||
**Depends on:** ADR-0091, ADR-0092, ADR-0093, ADR-0096
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
The mathematics/logic substrate is more complete than recent planning
|
||||
documents acknowledged:
|
||||
|
||||
- `language_packs/data/en_mathematics_logic_v1/` ships with manifest,
|
||||
lexicon, glosses.
|
||||
- `teaching/domain_chains/mathematics_logic_chains_v1.jsonl` exists.
|
||||
- `evals/elementary_mathematics_ood/` exists.
|
||||
- `docs/gaps.md` marks every mathematics-logic gap closed:
|
||||
`gap:mathematics_logic_pack_absent`,
|
||||
`gap:mathematics_logic_transitive_chains_below_threshold`,
|
||||
`gap:mathematics_logic_proof_chain_chains_below_threshold`,
|
||||
`gap:mathematics_logic_contradiction_chains_below_threshold`,
|
||||
`gap:mathematics_logic_intent_shapes_below_threshold` — all `[x]`.
|
||||
|
||||
What is missing is the formal ratification step under ADR-0091. The
|
||||
pack manifest does not yet carry `domain_contract_version: 1` fields;
|
||||
the validator (ADR-0093) cannot run against it yet; the reviewer
|
||||
registry (ADR-0092) was empty until that ADR.
|
||||
|
||||
This ADR is the first concrete domain claim under the Domain Pack
|
||||
Contract. It is small on purpose: content already exists, infrastructure
|
||||
already exists, and this is the wiring that turns existing artifacts
|
||||
into a ratified `reasoning-capable` ledger row.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
Ratify `en_mathematics_logic_v1` as `reasoning-capable` under
|
||||
ADR-0091 by emitting the contract fields into its manifest and passing
|
||||
`core capability domain-contract validate en_mathematics_logic_v1`.
|
||||
|
||||
### Manifest additions
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"domain_contract_version": 1,
|
||||
"domain_id": "mathematics_logic",
|
||||
"axioms": null,
|
||||
"rules": null,
|
||||
"teaching_chains": ["mathematics_logic_chains_v1"],
|
||||
"eval_lanes": [
|
||||
{
|
||||
"lane": "elementary_mathematics_ood",
|
||||
"version": "v1",
|
||||
"splits": ["dev", "public", "holdout"]
|
||||
},
|
||||
{
|
||||
"lane": "inference_closure",
|
||||
"version": "v1",
|
||||
"splits": ["dev", "public", "holdout"]
|
||||
},
|
||||
{
|
||||
"lane": "fabrication_control",
|
||||
"version": "v1",
|
||||
"splits": ["dev", "public", "holdout"]
|
||||
}
|
||||
],
|
||||
"reviewers": ["shay-j"],
|
||||
"known_gaps": [],
|
||||
"provenance": "adr-0097:reviewed:2026-05-21"
|
||||
}
|
||||
```
|
||||
|
||||
`axioms` and `rules` stay `null` at v1. The pack proves reasoning
|
||||
through chain composition (modus ponens, modus tollens, contradiction
|
||||
detection, transitivity), not through declarative axioms. A future
|
||||
ADR may add explicit axioms; this one does not.
|
||||
|
||||
### Required existing evidence (no new content)
|
||||
|
||||
- `mathematics_logic_chains_v1.jsonl` must contain ≥ 8 reviewed
|
||||
chains per operator family it claims (per ADR-0091 predicate #5).
|
||||
If it does not, ratification is blocked. Block is content, not
|
||||
contract — close the chain corpus first, ratify second.
|
||||
- ≥ 3 intent shapes present (per predicate #6). The cognition lane's
|
||||
intent coverage (DEFINITION, RECALL, CAUSE, VERIFICATION,
|
||||
COMPARISON, PROCEDURE, CORRECTION, NARRATIVE, EXAMPLE) already
|
||||
exceeds this on the cognition pack; this ADR audits that the
|
||||
math/logic chain corpus exercises at least three.
|
||||
- Three-split eval lane referenced. `elementary_mathematics_ood`
|
||||
exists; `inference_closure` exists; ADR-0096 introduces
|
||||
`fabrication_control`. Holdout paths verified non-empty.
|
||||
|
||||
### What this ADR does not do
|
||||
|
||||
- Does not introduce new lemmas, chains, or evals.
|
||||
- Does not promote to `expert-demo`. That requires audit-tour-equivalent
|
||||
reports — separate ADR.
|
||||
- Does not modify the runtime. Pack mounting behavior is unchanged.
|
||||
|
||||
---
|
||||
|
||||
## Invariant
|
||||
|
||||
`mathematics_logic_reasoning_capable_ledger_row` — running
|
||||
`core capability ledger` emits a row for `domain_id: mathematics_logic`
|
||||
with `status: reasoning-capable`, with provenance pointing at this
|
||||
ADR, and refuses to advance to `expert-demo` until a future ADR
|
||||
attaches the required reports.
|
||||
|
||||
---
|
||||
|
||||
## Lane
|
||||
|
||||
No new lane. Existing lanes carry the evidence:
|
||||
|
||||
- `elementary_mathematics_ood/` (positive coverage)
|
||||
- `inference_closure/` (composition)
|
||||
- `fabrication_control/` (negative control, ADR-0096)
|
||||
- `evals/domain_contract_validation/` (ADR-0093 confirms predicates fire)
|
||||
|
||||
CLAIMS.md Tier 2 gains one row pointing at the four lanes and this
|
||||
ADR.
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundary
|
||||
|
||||
Manifest edit is a reviewed proposal. Checksum refresh per
|
||||
ADR-0027/0029 discipline. No runtime mount path changes. No new
|
||||
filesystem writes outside the manifest and its checksum companion.
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
- First `reasoning-capable` row in the capability ledger backed by an
|
||||
ADR-0091-validated pack.
|
||||
- Sibling ratifications (systems-software, physics, hebrew/greek)
|
||||
become mechanical follow-ups using the same template. Each one is
|
||||
its own ADR but each is small.
|
||||
- The public showcase demo (ADR-0099) gains a real domain to draw
|
||||
Scene 4 from.
|
||||
|
||||
---
|
||||
|
||||
## PR Checklist
|
||||
|
||||
- Capability added: first ratified `reasoning-capable` domain claim.
|
||||
- Invariant proved: `mathematics_logic_reasoning_capable_ledger_row`.
|
||||
- Lanes proving it: four existing lanes plus ADR-0093 contract validator.
|
||||
- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none. Ratification is the reviewed proposal.
|
||||
- Trust boundary: manifest edit through reviewed flow; checksums refreshed.
|
||||
152
docs/decisions/ADR-0098-demo-composition-contract.md
Normal file
152
docs/decisions/ADR-0098-demo-composition-contract.md
Normal file
|
|
@ -0,0 +1,152 @@
|
|||
# ADR-0098 — Demo Composition Contract
|
||||
|
||||
**Status:** Proposed
|
||||
**Date:** 2026-05-21
|
||||
**Author:** CORE agents + reviewers
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
CORE has shipped a growing set of operator-facing demos:
|
||||
|
||||
- `core demo audit-tour` (ADR-0042)
|
||||
- `core demo anti-regression` (ADR-0055)
|
||||
- `core demo learning-loop` (ADR-0056)
|
||||
- `core demo register-tour` (ADR-0072)
|
||||
- `core demo anchor-lens-tour` (ADR-0073d)
|
||||
- `core demo orthogonality-tour` (ADR-0074)
|
||||
- `core bench --suite teaching-loop` (ADR-0057)
|
||||
|
||||
Each one is correct in isolation. Each one stands as its own evidence
|
||||
of a specific invariant. The problem the next ADR (ADR-0099 public
|
||||
showcase) faces is that there is no shared contract that lets one
|
||||
demo safely embed another.
|
||||
|
||||
Without a contract, the showcase has two bad options:
|
||||
1. Reimplement portions of each demo inline (drift, duplication,
|
||||
doctrine violation).
|
||||
2. Subprocess-spawn each demo and parse stdout (fragile, breaks the
|
||||
trace-hash discipline).
|
||||
|
||||
The right answer is a small protocol that the existing demos retrofit
|
||||
to, and that the showcase consumes.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
Introduce `DemoCommand` as a typed protocol. Existing demos are
|
||||
retrofitted to it in the same PR (mechanical, small). Future demos
|
||||
implement it from the start.
|
||||
|
||||
### Protocol
|
||||
|
||||
```python
|
||||
class DemoCommand(Protocol):
|
||||
demo_id: str # stable identifier, kebab-case
|
||||
claim_contract_version: int # currently 1
|
||||
|
||||
def run(self, *, output_dir: Path, seed: int | None = None) -> DemoResult: ...
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class DemoResult:
|
||||
demo_id: str
|
||||
claims: tuple[Claim, ...]
|
||||
evidence: Mapping[str, str] # claim_id -> evidence locator (path or sha)
|
||||
all_claims_supported: bool
|
||||
json_path: Path
|
||||
trace_features: Mapping[str, str] # canonical, for showcase composition
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class Claim:
|
||||
claim_id: str
|
||||
statement: str
|
||||
supported: bool
|
||||
evidence_locator: str
|
||||
```
|
||||
|
||||
### Rules
|
||||
|
||||
1. **Deterministic JSON.** Two runs with the same inputs and seed
|
||||
produce byte-identical `json_path` contents. HTML may differ
|
||||
in formatting; JSON is the truth-path.
|
||||
2. **No global state mutation.** A demo's `run()` may not mutate
|
||||
process-global registries (runtime singletons, telemetry sinks
|
||||
attached at module load, environment variables outside its own
|
||||
scope). Demos that need a telemetry sink attach a local one and
|
||||
detach it before returning.
|
||||
3. **Declared output paths only.** A demo writes only under
|
||||
`output_dir`. Path traversal rejected via `safe_pack_id`-class
|
||||
sanitization.
|
||||
4. **Composability is read-only.** A composing demo (the showcase)
|
||||
may read another demo's `DemoResult` but never mutates it.
|
||||
|
||||
### Retrofit scope
|
||||
|
||||
Each shipped demo gains a thin adapter in
|
||||
`core/commands/demo_<name>.py` that conforms to `DemoCommand`. The
|
||||
adapter does not change demo behavior; it wraps the existing entry
|
||||
point and produces a `DemoResult`.
|
||||
|
||||
### What this ADR does not do
|
||||
|
||||
- Does not change demo behavior.
|
||||
- Does not change demo CLI surface. `core demo audit-tour` runs the
|
||||
same way; the protocol is internal.
|
||||
- Does not introduce a registry. Demos remain discoverable via the
|
||||
existing CLI subparser.
|
||||
|
||||
---
|
||||
|
||||
## Invariant
|
||||
|
||||
`demo_composition_no_side_effects` — a grep gate on the showcase's
|
||||
import graph refuses any symbol that mutates runtime singletons or
|
||||
attaches telemetry sinks at module load. The protocol contract is
|
||||
enforced by structure, not by hope.
|
||||
|
||||
`demo_json_byte_equality` — for each demo retrofitted under this ADR,
|
||||
running it twice with identical inputs produces byte-identical JSON.
|
||||
CI lane verifies.
|
||||
|
||||
---
|
||||
|
||||
## Lane
|
||||
|
||||
`evals/demo_composition/` (new):
|
||||
|
||||
- positive: each retrofitted demo runs twice → identical JSON
|
||||
- negative: a deliberately stateful test fixture → composition
|
||||
detector rejects it
|
||||
- composition: showcase reads two demo results → produces composite
|
||||
claim set without mutating either
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundary
|
||||
|
||||
Demos write only to operator-specified `output_dir`. Path traversal
|
||||
rejection inherits from ADR-0051. No dynamic imports. No network. No
|
||||
shell.
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
- ADR-0099 public showcase becomes mechanically possible without
|
||||
reimplementing demo logic.
|
||||
- Future demos cost less: implement the protocol once, gain
|
||||
composability for free.
|
||||
- The shipped demos gain a small adapter layer but no behavioral
|
||||
change.
|
||||
|
||||
---
|
||||
|
||||
## PR Checklist
|
||||
|
||||
- Capability added: composition protocol for demos.
|
||||
- Invariants proved: `demo_composition_no_side_effects`, `demo_json_byte_equality`.
|
||||
- Lane proving it: `evals/demo_composition/`.
|
||||
- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none.
|
||||
- Trust boundary: demos write only under declared output paths; no global state mutation.
|
||||
156
docs/decisions/ADR-0099-public-showcase-demo.md
Normal file
156
docs/decisions/ADR-0099-public-showcase-demo.md
Normal file
|
|
@ -0,0 +1,156 @@
|
|||
# ADR-0099 — Public Showcase Demo
|
||||
|
||||
**Status:** Proposed
|
||||
**Date:** 2026-05-21
|
||||
**Author:** CORE agents + reviewers
|
||||
**Depends on:** ADR-0096, ADR-0097, ADR-0098
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
CORE's distinctive properties (deterministic cognition, honest
|
||||
refusal, reviewed learning, multi-hop composition with replayable
|
||||
trace) are each measured by a different shipped artifact. A first-time
|
||||
viewer has to read seven ADRs to assemble the thesis.
|
||||
|
||||
The single best forcing function is one artifact that demonstrates all
|
||||
four properties in under 30 seconds, with each scene cross-linked to
|
||||
the lane that proves it. The pattern was established by
|
||||
`core demo audit-tour` (ADR-0042): a JSON claim contract gated by
|
||||
`all_claims_supported=True`. This ADR generalizes that pattern across
|
||||
the four key invariants.
|
||||
|
||||
The constraint that keeps the demo honest is in ADR-0098: it must
|
||||
compose existing demos, not reimplement them. Every claim the showcase
|
||||
makes traces back to an already-passing lane.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
Introduce `core demo showcase` that composes four scenes, each
|
||||
delegating to a `DemoCommand` (ADR-0098). Output is a single
|
||||
deterministic JSON contract plus an HTML render.
|
||||
|
||||
### Scenes
|
||||
|
||||
**Scene 1 — Determinism.**
|
||||
Delegate: existing `register-tour` `DemoCommand` adapter.
|
||||
Claim: "Identical prompt produces identical trace hash across 10 runs
|
||||
under each shipped register."
|
||||
Evidence: `register-tour` JSON; `trace_hash` set cardinality = 1 per
|
||||
register.
|
||||
|
||||
**Scene 2 — Honest unknown.**
|
||||
Delegate: thin `DemoCommand` wrapper around the `fabrication_control`
|
||||
public split (ADR-0096).
|
||||
Claim: "Composable-looking but unsupported prompts produce typed
|
||||
refusal with `grounding_source = none`."
|
||||
Evidence: `fabrication_control` report SHA;
|
||||
`fabrication_rate ≤ 0.01`.
|
||||
|
||||
**Scene 3 — Reviewed learning.**
|
||||
Delegate: existing `learning-loop` `DemoCommand` adapter.
|
||||
Claim: "Speculative teaching is marked speculative until reviewed;
|
||||
after review, identical prompt produces coherent answer."
|
||||
Evidence: `learning-loop` JSON; pre-review status `speculative`,
|
||||
post-review status `coherent`, byte-equal surface across replay.
|
||||
|
||||
**Scene 4 — Multi-hop with trace.**
|
||||
Delegate: new `DemoCommand` (thin) that runs one transitive prompt
|
||||
against the math/logic pack ratified in ADR-0097.
|
||||
Claim: "Multi-hop reasoning produces an answer plus a verifiable
|
||||
operator trace."
|
||||
Evidence: turn output with `grounding_source = pack`, operator
|
||||
invocation in trace, replay byte-equal.
|
||||
|
||||
### Output contract
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"showcase_version": 1,
|
||||
"generated_at_revision": "<git sha>",
|
||||
"scenes": [
|
||||
{
|
||||
"scene_id": "determinism",
|
||||
"demo_id": "register-tour",
|
||||
"claims": [...],
|
||||
"all_claims_supported": true
|
||||
},
|
||||
...
|
||||
],
|
||||
"all_claims_supported": true,
|
||||
"total_runtime_ms": ...,
|
||||
"trace_hash": "..."
|
||||
}
|
||||
```
|
||||
|
||||
### Hard constraints
|
||||
|
||||
- **Total runtime <30s on dev hardware.** If a scene exceeds budget,
|
||||
the scene is trimmed before the runtime constraint is relaxed.
|
||||
- **No new mechanism.** Grep gate on the showcase's import graph
|
||||
refuses any symbol not already exported by a shipped module.
|
||||
- **JSON byte-equality across runs.** HTML may vary in styling; JSON
|
||||
must not. Replay verified in CI.
|
||||
- **Public-safety preserved.** Every surface emitted is already
|
||||
emitted by an existing public demo. No new exposure.
|
||||
|
||||
---
|
||||
|
||||
## Invariant
|
||||
|
||||
`public_showcase_pure_composition` — grep gate refuses any symbol in
|
||||
the showcase's import graph not already exported by `core/`, `chat/`,
|
||||
`generate/`, `language_packs/`, `teaching/`, or `core.commands.demo_*`.
|
||||
|
||||
`public_showcase_all_claims_supported` — CI fails if showcase exits
|
||||
with `all_claims_supported=False` or runtime >30s.
|
||||
|
||||
`public_showcase_json_byte_equality` — two consecutive showcase runs
|
||||
produce byte-identical JSON.
|
||||
|
||||
---
|
||||
|
||||
## Lane
|
||||
|
||||
`evals/public_demo/` (existing directory, populated by this ADR):
|
||||
|
||||
- determinism: showcase JSON byte-equal across two runs
|
||||
- support: `all_claims_supported=True`
|
||||
- budget: total runtime <30s on dev hardware reference machine
|
||||
- composition: grep gate confirms pure composition
|
||||
- scene-level: each scene's underlying demo lane also passes
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundary
|
||||
|
||||
Showcase writes under operator-specified `--output-dir`. Path traversal
|
||||
rejection per ADR-0051 / ADR-0098. No network. No shell. No dynamic
|
||||
imports. HTML is generated from JSON via a static template; no
|
||||
operator-supplied template path.
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
- One artifact answers "what makes CORE distinct" in under 30 seconds.
|
||||
- Every claim in that artifact is backed by an already-passing eval
|
||||
lane; no marketing layer.
|
||||
- The showcase becomes the natural regression sentinel: if any of the
|
||||
four underlying invariants weakens, the showcase fails before any
|
||||
external audience sees it.
|
||||
- ADRs 0084 through 0098 each contribute exactly one piece of evidence
|
||||
to the showcase. The slate closes.
|
||||
|
||||
---
|
||||
|
||||
## PR Checklist
|
||||
|
||||
- Capability added: single artifact composing four CORE invariants under 30s.
|
||||
- Invariants proved: `public_showcase_pure_composition`, `public_showcase_all_claims_supported`, `public_showcase_json_byte_equality`.
|
||||
- Lane proving it: `evals/public_demo/`.
|
||||
- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none. Pure composition enforced by grep gate.
|
||||
- Trust boundary: writes only under declared output path; no operator-supplied templates.
|
||||
Loading…
Reference in a new issue