Commit graph

147 commits

Author SHA1 Message Date
Shay
a36b48b198
feat(runtime): opt-in unified-ingest path (ADR-0090, audit Findings 6+7) (#95)
Closes audit Findings 6 (within-turn recall not batched) and 7
(probe-ingest / commit-ingest dual field) as a single PR — the two
are architecturally entangled and resolve together.

Pre-fix flow in ``ChatRuntime.chat()``:

  1. ``probe_ingest(filtered)`` → ``probe_state.F``
  2. Gate check on ``probe_state.F``
  3. If gate fires: ``commit_ingest`` + stub response
  4. Otherwise: ``commit_ingest`` + drive bias → ``field_state.F``
  5. Walk runs on ``field_state.F``

The gate observes one manifold position; the walk navigates a
slightly different one (drive bias applied between them).  Honest
refusal decisions and walk outputs are made on different fields —
the audit's named coherence gap.

This PR ships a flag-gated unified-ingest path following the
codebase's standard substantive-change pattern (ADR-0046 /
ADR-0062 / ADR-0085 / ADR-0088 / ADR-0089):

``RuntimeConfig.unified_ingest: bool = False`` (default).

When ``True``:

  1. ``commit_ingest(filtered)`` runs first.
  2. Drive bias applied immediately.
  3. Gate observes ``committed.F``.
  4. If gate fires: stub response (turn has already committed —
     intentional semantic change documented in ADR-0090).
  5. Otherwise: walk runs on the same ``committed.F`` the gate
     decided against — no second ``commit_ingest`` call.
  6. ``probe_ingest`` is not called on this path.

When ``False`` (default): historical behavior is preserved
bit-for-bit; ``probe_ingest`` still runs first.

ADR-0090 documents:

  * Phase 1 (this PR): unified-ingest substrate.
  * Phase 2 (separate PR, after Phase 1 validates): batched recall
    — pass the gate's ``direct_hits`` into ``generate()`` as a
    ``prebuilt_first_recall`` so the walk's first step does not
    re-call ``vault.recall()`` on the same field.  Single recall
    call eliminated per turn.
  * Out of scope: ``recall_batch`` for per-step walk recalls
    (each step's query depends on the previous step's field
    state; not batchable without changing walk geometry).

Validation:

  * 5 new tests in ``tests/test_unified_ingest_null_lift.py``:
      - flag defaults to ``False`` on ``DEFAULT_CONFIG``
      - flag-off surface + trace_hash + vault_hits byte-identical
      - flag-on does not call ``probe_ingest`` (verified via spy)
      - flag-on produces well-formed surface + trace_hash
      - flag-off still calls ``probe_ingest`` (historical guard)
  * ``core eval cognition`` byte-identical across all three splits:
    public 100/100/91.7/100, dev 100/100/78.6/100, holdout
    100/100/83.3/100.
  * ``core test --suite cognition`` 120/0/1, ``smoke`` 67/0,
    ``runtime`` 19/0.

Comb-pass status after this PR:

  * Item 4 (graph topo) ✓ #92
  * Item 5 (realizer node_map) ✓ #91
  * Item 6 (batch recall) ✓ ADR-0090 substrate (this PR); Phase 2
    optimization is queued
  * Item 7 (probe/commit dual ingest) ✓ ADR-0090 (this PR)
  * Item 8 (dead defensiveness sweep) ✓ #91
  * Item 9 (local imports) ✓ #91
  * Item 11 (dead ``_fold_compose_into_surface``) ✓ #91
  * Item 13 (``_serialize_*`` fold) ✓ #91
  * Item 15 (GenerationResult tuple/list) ⊘ false positive
  * Item 16 (subject normalization consistency) ✓ #93
  * Item 17 (redundant ``^`` anchors) ✓ #94
  * Tier 5 minor (``_BE_FORMS`` hoist, walrus, reverse-iter) ✓ #94
2026-05-20 21:00:27 -07:00
Shay
5446ab1615
docs: pack inventory 2026-05-21 — strengths, weaknesses, leverage gaps (#84)
Comprehensive survey of every pack in the tree across five layers
(primitives → language → teaching → policy → selection axes → style)
with per-pack stats, cross-pack lemma overlap, teaching-chain graph
topology, and a ranked top-leverage-gaps list.

Key findings:

  * **Layer 4 (selection axes) is the densest by far** — 24 packs
    (17 anchor lenses + 7 registers).  Greek and Hebrew lens
    families are at parity (8 each).
  * **Layer 2 (teaching corpora) is the load-bearing thin layer** —
    only 41 reviewed chains across 18 subjects, with just 2 intent
    shapes covered (CAUSE / VERIFICATION) and 7 connectives total.
    No COMPARISON / PROCEDURE / CORRECTION chains exist; those
    intents route through pack composers only.
  * **Layer 1 EN is solid** — 354 lemmas across 12 mounted packs
    with 93% gloss coverage; 24 ungloss'd lemmas remain.
  * **Cross-language is structurally present but content-light** —
    grc / he micro-packs ship and mount; the bigger
    ``*_cognition_v1`` siblings ratify but are not in the default
    mount; no glosses exist on any non-EN pack.
  * **Three drafted ethics packs are unratified** (legal /
    research / engineering) — the highest-leverage low-effort gap.
  * **Rhetorical-style axis ships substrate only** — one
    null-lift pack; ADR-0087 consumer phase pending.

Top-leverage gaps ordered by ratio of unblocked value to effort
are enumerated in §7 of the doc itself.  No code lands here.
2026-05-20 19:59:18 -07:00
Shay
8b51813c73
docs(adr): ADR-0088 + ADR-0089 — unblock Findings 2 + 4 (#81)
Two ADRs that unblock the remaining items from the 2026-05-20 audit
that could not ship as direct PRs.

ADR-0088 — Realizer-Grounded Authority (Finding 2 retry)
========================================================

The first-response audit remedy (wire ``ground_graph`` between
``runtime.chat`` and ``realize_semantic``) was empirically attempted
on ``fix/ground-graph-wiring`` and reverted: the grounded realizer's
template output (e.g. ``"Light is a visible medium that reveal
truth"``) is grammatically and stylistically weaker than the runtime
path's ADR-0085-polished pack-grounded surface, so the realizer wins
the surface resolver (PR #76) and the user-visible surface regresses
on 23 byte-identical tests + ``register_invariant_grounding``.

ADR-0088 reframes Finding 2 as a two-phase rollout:

  * Phase A (no behavior change) — realizer fluency parity.
    Templates consult the same gloss source ADR-0085 wired into the
    CAUSE composer, emit 3sg verb agreement, and carry the same
    pack-provenance tag the runtime path emits.  Byte-identical
    today because the realizer is still gated by
    ``_is_useful_surface``.
  * Phase B (substantive) — ground the graph and let the realizer
    compete.  Surfaces change exactly once, with a per-case
    re-baseline justified by a "fluency ≥ pre-fix runtime surface"
    invariant.

The audit's final-draft remedy (hot-path short-circuit only) is
explicitly rejected — pure perf cleanup, no metric lift since
``core eval cognition`` is already at 100% groundedness.

ADR-0089 — Compound-Intent Pipeline Dispatch (Finding 4)
========================================================

``classify_compound_intent`` is implemented but never reaches
``CognitiveTurnPipeline.run()``.  Compound inputs like *"What is X
and how does it relate to Y?"* silently drop the second clause.

Naive multi-node dispatch breaks every downstream stage:
PropositionGraph (one root), plan_articulation (single-root),
realize_semantic (one target), surface resolver (one surface per
turn), compute_trace_hash (one intent_tag + articulation_surface),
teaching loop (one correction-source proposal), register / anchor-
lens telemetry (one variant per turn).

ADR-0089 proposes a three-phase rollout:

  * Phase C1 (no behavior change) — call ``classify_compound_intent``
    in step 1b, record dropped clauses on
    ``CognitiveTurnResult.dropped_compound_clauses`` for
    observability while routing the dominant clause through the
    existing single-intent path.
  * Phase C2 (opt-in substantive) — flag-gated multi-node graph
    dispatch with new ``CompoundEdge`` / ``ConjunctionRelation``,
    widened ``compute_trace_hash`` carrying a
    ``compound_clauses_hash``, and a ``multi_clause_surface`` field
    on the resolver.  Flag-off preserves byte-identity.
  * Phase C3 — telemetry alignment + demo + docs.

Each phase is independently shippable and preserves the existing
null-lift / byte-identity invariants register and anchor-lens
established (ADR-0072, ADR-0073d) as the project's pattern for
substantive runtime behavior changes.

Both ADRs are Proposed; ratification follows the existing pack /
ADR review process.  No code lands in this commit.
2026-05-20 19:59:14 -07:00
Shay
886fc4553f
docs(adr-0085): content style pass v2 brief — 3sg + plural agreement (#75)
Follow-up brief for the cheaper dev agent.  Scoped tight: ~18 concrete
row-by-row edits across two patterns the v1 pass (PR #73) deferred or
missed.

Pattern A — 3sg present-tense agreement after relative pronouns
  what/who/that/which followed by a bare-form verb where the implied
  subject is singular.  10 candidates identified up front by repo scan:
    - en_core_causation_v1/effect       (with judgment note)
    - en_core_cognition_v1/beginning, creation, definition, evidence,
      light, reason, symbol
    - en_core_meta_v1/example, mind

  Brief explicitly clarifies that bare verbs after modals (`can`,
  `may`, `should`) are CORRECT and must NOT be changed — flagging
  the "who can know and do" case the v1 agent did right.

Pattern B — plural agreement after quantifier / preposition
  between/among/of/two/three/many + count-noun-in-singular.  5
  confident edits + 3 borderline cases with judgment guidance:
    - en_core_attitude_v1/broad
    - en_core_cognition_v1/context, order, style
    - en_core_spatial_v1/between
  Borderline guidance distinguishes count vs mass nouns: `reason`
  in "group of reason" is count (apply fix); `reason` in "because
  of reason" is mass (leave alone).

Same hard rules as v1 brief:
  - no code edits
  - definitional_atoms / predicates_invited / pos / lemma /
    definition_version must not change
  - Greek/Hebrew packs and primitives pack untouched
  - closure verifier must exit 0
  - cognition eval must stay byte-identical
  - draft PR, human review before merge

Estimated effort: ~18 one-character edits.  Whole pass should take an
order of magnitude less time than v1 because the candidate rows are
enumerated in the brief itself rather than discovered via heuristic
scan.

Why a v2 brief rather than amending v1's PR:
  Plural patterns were not on v1's explicit pattern list (the v1
  brief only named verb agreement, missing articles, missing
  infinitives, missing copulas).  3sg agreement was named but
  required a different tool than the agent had at hand.  Scoping
  v2 to exactly the rows known to need fixing is cheaper than
  re-running the v1 heuristic scan with an expanded ruleset.

Provenance tag for the v2 pass:
  adr-0085-style-v2:reviewed:2026-05-22
2026-05-20 16:23:14 -07:00
Shay
ac75cfc659
docs(adr-0087): rhetorical style as selection axis + writing-chain harvester spec (#72)
Pre-work for a writing-curriculum extension to CORE.  Two companion
documents, both Proposed status (no code shipped).

docs/decisions/ADR-0087-rhetorical-style-axis.md
  Pins rhetorical style as a third selection axis — sibling to anchor
  lens (ADR-0073), orthogonal to register (ADR-0070).  Substantive
  axis: trace_hash DISTINCT across styles (style changes which moves
  the composer requires and which frames the realizer emits, which
  changes the propositional plan, which changes the trace).

  Four anti-patterns explicitly named and rejected:
    - style as motor (re-couples realizer to geometry; same shape as
      the ADR-0085 fusion-operator rejection)
    - style as register variant (conflates substantive with stylistic)
    - style as identity axis (bloats identity doctrine)
    - style auto-detected from user input (operator-chosen only)

  Pack shape mirrors packs/anchor_lens/.  default_unstyled_v1 is the
  null-lift pack identical to no-style behavior.  Three CI invariants
  proposed: rhetorical_style_null_lift, schema validation, three-axis
  orthogonality.

  Substrate-only ADR — no consumer code, no genre packs.  Consumer
  integration is a follow-up ADR (composer + realizer extensions
  that read permitted_frames + required_moves_per_claim +
  forbidden_moves).

docs/curriculum/writing-chain-harvester-spec.md
  Layer 0 of the writing curriculum.  A deterministic tool that
  extracts candidate (subject, predicate, object) triples from
  reviewed expert prose and surfaces them as proposals to the
  existing teaching/review pipeline.

  Five stages (segment → classify → extract → propose → audit) —
  pure-Python rule-based, no LLM generation, no auto-acceptance.
  Trust boundary: reviewer accept/reject via the existing
  core teaching propose/review path.  No bypass permitted.

  The harvester is a proposal PRODUCER, not a proposal CONSUMER.
  Plugs into the existing pipeline without inventing a new review
  mechanism.  Each proposal carries source_id + source_line + the
  exact source_clause it came from for reviewer verification.

  First-implementation acceptance criteria deliberately tight:
  Stage 0+1 with dry-run only.  Stages 2-5 are follow-up PRs.

Substrate-first sequencing pattern (ADR-0084 → 0085) reused
throughout.  Both documents acknowledge open questions deferred to
implementation phase rather than pre-deciding.

Why now: a writing curriculum is being scoped.  Without this ADR,
every downstream PR faces the same "should style be a motor?"
question and the temptation to reach for the geometry will recur
every time the realizer produces a stilted surface.  Pinning the
axis up front prevents that recurrence.
2026-05-20 16:09:16 -07:00
Shay
537d73e394
docs(adr-0085): content style pass brief for the cheaper dev agent (#71)
Brief for a fluency pass on the 333 ratified gloss entries. Closes the
content-side counterpart of ADR-0085's surface lift:

  Before  "Light exists as visible medium that reveal truth."
  After   "Light exists as a visible medium that reveals truth."

Same gloss content, English-correct. Fixes 3sg agreement after
relative clauses, plural agreement after quantifiers, missing
articles before adjective-noun NOUN-frame glosses, and missing 'to'
in VERB-frame glosses.

Hard constraints encoded in the brief (matching the wrapper-prompt
pattern that worked for ADR-0084 content):

- code untouched: only language_packs/data/<pack>/glosses.jsonl edits
- definitional_atoms, predicates_invited, pos, lemma, definition_version
  must NOT change
- closure verifier (scripts/verify_definitional_closure.py) must
  still exit 0
- cognition eval must stay byte-identical to baseline
- Greek/Hebrew packs untouched (not in definitional layer per ADR-0084
  scope limit)
- primitives pack untouched
- draft PR, human review before merge

Includes a Phase-1 inventory script the agent runs first to scope the
work (heuristic pattern matcher across the 13 opted-in packs),
worked-example fluency rules with before/after table, per-pack
checksum-refresh shell snippet, and Phase-4 verification commands.

Estimated effort: ~30-60 lines of JSONL edits. Same handoff format as
docs/handoff/ADR-0084-pack-content-brief.md.
2026-05-20 15:58:44 -07:00
Shay
4b9404a88e
feat(adr-0085): gloss-aware CAUSE composer — explanation frame from glosses (#70)
The original "Why does light exist?" complaint that motivated ADR-0084
was specifically about CAUSE-intent surfaces. ADR-0084 (substrate) +
PR #65 (content) already moved DEFINITION/RECALL to gloss-grounded
surfaces ("Light is visible medium that reveal truth."). But CAUSE
still dispatched through the chain-walk path:

  Before: light — teaching-grounded (cognition_chains_v1):
            cognition.illumination; logos.core.
            light reveals truth (cognition.truth).
            No session evidence yet.

  After:  Light exists as visible medium that reveal truth.
          pack-grounded (en_core_cognition_v1).

The chain-walk is structurally correct but the wrong SHAPE for a why-
question — it's a graph traversal, not an explanation. ADR-0085 fixes
the shape using the same gloss material that DEFINITION/RECALL already
consume, with no new content authoring.

Additive composer
  chat/pack_grounding.py:gloss_aware_cause_surface()
  - Resolves gloss via lexicon-residency-checked resolve_gloss().
  - Frames POS-aware:
      NOUN -> "{Lemma} exists as {gloss}."
      VERB -> "To {lemma} is to {gloss}."
      ADJ  -> "To be {lemma} is to {gloss}."
      *    -> falls back to _frame_gloss (predicate-identity).
  - Threads anchor lens via the existing helper (ADR-0073c parity).
  - Returns None when no gloss exists — runtime falls through to the
    existing chain-walk path. Additive: no CAUSE case loses its surface.

Runtime dispatch
  chat/runtime.py — IntentTag.CAUSE tries gloss path FIRST under the
  flag; falls through to teaching_grounded_surface* on None.
  Unconditional fallback — never silent.

Opt-in flag
  core/config.py — RuntimeConfig.gloss_aware_cause: bool = False
  Default off preserves pre-ADR-0085 chain-walk surfaces byte-
  identically (null-drop invariant, CI-pinned).

Prompt-diversity classifier update
  evals/prompt_diversity/runner.py — _CAUSE_MARKERS widened with the
  explanation-frame markers ("exists as", "is to", "to be", "is for",
  "purpose of") plus bare-form predicates ("reveal" alongside
  "reveals"). Neither composer path is penalised on shape_fit just on
  inflection grounds.

v1/public lift (flag OFF vs ON, 26 cases)
  intent_accuracy        : 65.4% -> 65.4%   ( — )
  versor_closure_rate    : 100.0% -> 100.0% ( — )
  response_shape_fit     : 57.7% -> 57.7%   ( — , both frames recognized)
  audit_in_surface_rate  : 42.3% -> 42.3%   ( — , envelope ADR's job)
  gloss_quote_rate       : 11.5% -> 23.1%   (+11.5pp, structural lift)

Tests (15)
  - 5 pure composer (NOUN/VERB frame, unknown/empty None, no chain-
    walk artifacts in surface)
  - 5 runtime dispatch (flag-off chain-walk, flag-on gloss, parametrized
    across glossed subjects, VERIFICATION unchanged under flag, no-
    gloss fallback engages)
  - 5 cognition lane invariance (aggregate metrics byte-identical
    under both flag states; surfaces deliberately shift on the 2 CAUSE
    cases with glossed subjects — the structural-change-vs-metric-
    invariance both-sides invariant)

Lanes
  smoke 67/0, cognition 120/0/1 skipped, packs 6/0, teaching 17/0,
  runtime 19/0. core eval cognition byte-identical 100/91.7/100/100
  under both flag states.

Scope limits (per ADR §Scope limits)
  - CAUSE only; VERIFICATION still chain-walks (different shape).
  - English pilot only; Greek/Hebrew packs not opted into definitional
    layer yet (ADR-0084 scope limit).
  - Single-lemma subjects; compound/anaphoric fall through.
  - Opt-in until cognition holdout confirms the lift transfers off-
    fixture. Future PR flips default on.

Out of scope
  - Surface-vs-envelope cleanup ("pack-grounded (...)" still leaks).
  - Predicate licensing (ADR-0086).
  - Content style pass (bare lemma forms in glosses — separate brief).
2026-05-20 15:55:08 -07:00
Shay
48282eef8d
feat(adr-0084): definitional layer — proposal + substrate (schema/loader/closure) (#64)
* docs(adr-0084): propose definitional layer + prompt-diversity suite

Three companion artifacts proposing the next substantive design step
after ADR-0083:

1. ADR-0084 (Proposed) — Definitional Layer for Lexicon Packs
   Optional `definition` block on pack entries: gloss,
   definitional_atoms, predicates_invited, definition_version,
   provenance.  Pack-level opt-in.  Closure rule: every word in a
   gloss must resolve to a same-pack lemma, another mounted pack's
   lemma, or a primitive in a new `packs/primitives/` pack.
   NO composer change in this ADR (sequenced for ADR-0085) —
   ratify substrate before any consumer depends on it.

2. evals/prompt_diversity/ (Proposed) — companion eval lane
   ~50 cases across question-shape × sophistication × domain,
   measuring three new metrics: response_shape_fit,
   audit_in_surface_rate (quantifies the trust-boundary leak into
   user surfaces), gloss_quote_rate (zero today; rises with future
   gloss-aware composer).  No v1 pass thresholds — the lane
   establishes a baseline distribution so future work has
   something to move.  26 seed cases authored covering all 21
   categories.

3. docs/handoff/ADR-0084-pack-content-brief.md — paste-ready brief
   for a cheaper/faster dev agent to produce the pack content in
   parallel.  Self-contained, 5 sequenced phases (primitives pack
   → extend 9 existing glosses → add to relations/anchors → write
   closure verifier → run safety lanes), explicit don't-touch list
   (no composer / runtime / algebra / Greek+Hebrew packs / schema
   parser), no-LLM-glosses discipline, per-phase acceptance.

Discovery while drafting: 9 packs already carry glosses.jsonl
under language_packs/data/ with a flat schema (78 entries in
en_core_cognition_v1 alone).  The brief reflects that — most
work is extending existing entries, not authoring from scratch.

Strategic context: ADR-0083 raised the *depth* ceiling on chain
composition; ADR-0084 raises the *fidelity* ceiling.  The φ
separation probe (memory: phi-separation-falsified) established
that semantic capability lives in chain composition, not in φ
geometry, so deepening the composer's substrate is the natural
next step.  ADR-0084 → 0085 (gloss-aware composer) → 0086
(predicate licensing at ratification) is the planned sequence.

* feat(adr-0084): substrate — schema parser, primitives loader, closure verifier

Substrate-only code-side for ADR-0084 (Definitional Layer for Lexicon Packs).
No composer touches the new fields yet; consumer integration is ADR-0085.

Schema (additive, default preserves byte-identity)
  - LanguagePackManifest.definitional_layer: bool = False
  - compiler loader propagates the flag from manifest.json

language_packs/definitions.py (new)
  - GlossEntry dataclass: lemma, gloss, pos, definitional_atoms,
    predicates_invited, definition_version, provenance_ids
  - parse_gloss_entry(payload, *, strict) — strict mode enforces ADR-0084
    §Schema validation row-by-row: required keys, typed lists, no
    unknown keys, positive definition_version; lax mode preserves the
    legacy two-field shape for back-compat
  - load_pack_glosses(pack_id, *, strict) with cache + clear hook
  - verify_definitional_closure(pack_id, *, mounted_pack_lemmas,
    primitive_lemmas, strict) returning tuple[ClosureViolation, ...];
    case-insensitive resolution; cycles permitted per ADR

packs/primitives/loader.py (new)
  - Sister loader to packs/safety/ and packs/identity/
  - PrimitivesPack frozen dataclass with .lemmas frozenset
  - Gates: checksum match, kind=='primitives', definitional_layer:true,
    never_auto_mutable:true, pack_id matches dir, primitive_count
    cross-check, duplicate-lemma rejection, path-traversal rejection,
    strict per-entry schema with allow-list
  - DEFAULT_PRIMITIVES_PACK = 'en_semantic_primitives_v1'

tests/test_adr_0084_definitional_substrate.py
  - 38 tests covering strict parser (each required key rejection, unknown
    key rejection, empty predicates_invited allowed, empty
    definitional_atoms rejected, invalid definition_version), lax
    parser back-compat, load_pack_glosses (missing/strict raise/lax
    skip/malformed JSON), closure verifier (same-pack/primitive/mounted/
    unresolved/case-insensitive), primitives loader (every gate), and
    a back-compat check that every shipped pack still ratifies with
    definitional_layer=False

Lanes: smoke 67/0, cognition 120/0/1, teaching 17/0, runtime 19/0,
packs 6/0. Cognition eval byte-identical 100/91.7/100/100.

When the content PR lands (primitives.jsonl + extended glosses.jsonl
under ADR-0084-pack-content-brief.md), the gate catches any closure-rule
violation without further code change.

* feat(evals): prompt_diversity lane runner — measurement instrument for ADR-0084+

Implements the runner against the existing contract.md + 26-case v1
public split.  Lane auto-discovered by evals.framework via the standard
contract + runner convention.

Runner (evals/prompt_diversity/runner.py)
  - run_lane(cases, *, config, workers) -> LaneReport
  - 5 metrics: intent_accuracy, versor_closure_rate (carried over from
    cognition), plus the three new lane-specific metrics —
    response_shape_fit, audit_in_surface_rate, gloss_quote_rate
  - breakdown dict groups by (question_shape, sophistication, domain)
    per contract §How to read the output
  - mirrors evals.cognition.runner's parallel worker pattern

Per-shape classifier (deliberately substring/regex-simple at v1)
  - predicate_identity, explanation, sequence, two_subject_contrast,
    narrative, honest_disclosure
  - Unknown shape => neutral pass (don't penalise new categories)

Audit-leak detector
  - trust-boundary preamble markers (teaching-grounded (, pack-grounded
    (, No session evidence yet.)
  - dotted semantic-domain tag regex (cognition.illumination, etc.)

Gloss-quote detector
  - resolves expected_terms via chat.pack_resolver.resolve_gloss
  - 4-token contiguous-window match against surface (high-confidence
    "gloss actually quoted", not "shared one common word")

Tests (tests/test_prompt_diversity_runner.py — 23)
  - shape classifier parametrized over the six expected_shape values
  - audit-leak detector parametrized over preamble + tag + clean cases
  - end-to-end on v1 public:
      * versor_closure_rate == 1.0 (only v1 pass threshold per contract)
      * every metric in [0, 1]
      * breakdown groups present with the four per-cell metrics
      * diversity gate: >=5 question shapes, >=3 domains
        (defends against future regressions that collapse the suite
         back to a cognition-shaped fixture)

v1/public baseline (26 cases)
  intent_accuracy      : 65.4%   (contract predicted 70-85%)
  versor_closure_rate  : 100.0%  (only v1 pass threshold)  PASS
  response_shape_fit   : 53.8%   (contract predicted low)
  audit_in_surface_rate: 42.3%   (contract predicted ~100%)
  gloss_quote_rate     :  7.7%   (contract predicted 0%)

Three baseline surprises worth noting in the report (NOT failures —
the v1 lane is explicitly there to establish the distribution):

  - audit_in_surface_rate at 42% (not 100%) means the chain-walk leak
    fires on ~11/26; the other 15 are honest-disclosure cases that
    emit no audit envelope.  Sharpens the future surface-vs-envelope
    ADR's actual target: grounded surfaces specifically.
  - response_shape_fit at 54% (not "low") — classifier likely has
    false positives on the ", which " cause-marker.  Worth tightening
    once we have an ADR-0085 baseline to compare against.
  - intent_accuracy at 65% (below predicted 70-85%) — classifier dips
    harder on adversarial/cross-pack than expected.  Real gap.

All five smoke/cognition/teaching/runtime/packs lanes still green;
core eval cognition byte-identical 100/91.7/100/100.

* feat(packs): ADR-0084 pack content (primitives + extend glosses + closure verifier) (#65)

* feat(packs): ADR-0084 pack content

* feat(packs): repair ADR-0084 definitional content

* test(adr-0084): adjust substrate manifest tests for post-#65 content reality

PR #65 flipped definitional_layer:true on 13 English packs (9 core +
4 relations + collapse-anchors).  The substrate's previous test
test_existing_packs_unchanged asserted that en_core_cognition_v1 +
en_core_relations_v1 still had definitional_layer:False — which was
the right pre-content invariant but is wrong post-content.

Replace it with two complementary tests that hold against real content:

  - test_non_opted_packs_default_false:
      pins that packs that DIDN'T flip the flag (en_minimal_v1,
      he_core_cognition_v1, grc_logos_cognition_v1) still surface
      definitional_layer=False through the loader.  Defends against
      a future change accidentally flipping the flag on a non-opted
      pack.

  - test_opted_packs_carry_flag:
      pins that packs that DID flip the flag (en_core_cognition_v1,
      en_core_relations_v1) surface definitional_layer=True through
      the loader.  Proves the substrate's manifest-field propagation
      works against real ratified content, not just fixture packs.

Net: +1 test, same intent (substrate ratifies the manifest field
correctly), now with real-content coverage on both sides of the gate.

All 62 ADR-0084 substrate + prompt-diversity tests pass.
2026-05-20 15:25:25 -07:00
Shay
9e6fa4be75
feat(adr-0083): transitive (multi-hop) teaching-grounded surface (#63)
Strict superset of ADR-0062's depth-1 composer.  `max_depth` is the
number of follow-up hops appended beyond the initial chain:

  max_depth=0  → byte-identical to single-chain surface
  max_depth=1  → byte-identical to ADR-0062 composed
  max_depth=2  → byte-identical to ADR-0062 when no second hop
                 survives, strict superset when one does

The composer surfaces content the realizer was silently dropping
from chains already ratified in `cognition_chains_v1`.  Example
live lift on `"Why does light exist?"`:

  composed: "light reveals truth, which grounds knowledge."
  transitive(2): "...which grounds knowledge, which requires evidence."

Cycle-safe at every depth via a single visited-set; single-corpus
traversal in v1 (cross-corpus transitive deferred to a follow-up
ADR alongside ADR-0064's cross-pack model).

Both flags default False — every existing surface is preserved
byte-identically.  When both `composed_surface` and
`transitive_surface` are True, transitive wins.

Implementation:
- `core/config.py`: `transitive_surface: bool = False`,
  `transitive_max_depth: int = 2`.
- `chat/teaching_grounding.py`: `_resolve_followup` shared helper
  refactored out of the depth-1 composer (no behavioural change),
  plus new `teaching_grounded_surface_transitive(subject,
  intent_tag, *, max_depth)`.
- `chat/runtime.py`: dispatch order — transitive > composed > single.

Verification:
- tests/test_transitive_surface.py: 16 new tests covering pure-fn
  contract, visited-set cycle guard at every depth, runtime
  integration, and the cognition-lane null-drop invariant at
  `max_depth=2` (public + holdout splits).
- tests/test_composed_surface.py: 11/11 pass after the helper
  refactor (ADR-0062 behaviour preserved).
- `core test --suite smoke`: 67 pass.
- `core test --suite cognition`: 120 pass, 1 skipped.
- `core test --suite teaching`: 17 pass.
- `core eval cognition`: 100 / 91.7 / 100 / 100 (byte-identical).
2026-05-20 14:11:40 -07:00
Shay
8f1903e8e7
chore(evals): contracts + bench json + Lane B viewer + chart + audit + demo schema (#62)
* chore(evals, cli): contract standardization + bench --json stdout cleanliness

End-of-session shippability pass.  Three concrete fixes:

1. core/cli.py — bench --json no longer pollutes stdout
   Several bench paths call scripts.run_pulse.run_pulse which prints
   verbose [pulse] traces unconditionally to stdout, breaking jq /
   programmatic consumers of --json output.

   New _bench_stdout_guard() redirects stdout → stderr for the
   duration of the bench run when --json is set.  Operator still sees
   the pulse trace (on stderr), but --json consumers get a clean JSON
   document on stdout.  Applied to all four bench paths: cost,
   articulation, default suite, and --suite all.

   Verified: core bench --suite determinism --json now produces
   parseable JSON; human path still shows 1140 [pulse] lines.

2. evals/{frontier_compare,realizer_guard}/contract.md (new)
   core/contemplation/contract.md (new)

   Each new contract follows the established pattern (37 contracts
   already exist under evals/<lane>/contract.md):

     - What it measures
     - Why it matters (structural win)
     - How to run
     - How to read the output
     - Pass criteria table
     - When it has failed and why
     - Runner / module layout

   Coverage:
     - frontier_compare: both Lane A (CORE-only suites) and Lane B
       (cross-provider prompt_battery) with explicit guardrails
       against mixing — operator asks for the wrong lane combination,
       runner exits 2 with helpful error.
     - realizer_guard: C1/C2 articulation safety boundary — synthetic
       illegal candidates rejected directly by check_surface AND
       former-bug runtime prompts now produce legal articulations.
     - contemplation (ADR-0080): not under evals/ since it's runtime
       infrastructure that consumes eval reports — contract lives at
       core/contemplation/contract.md.  Documents the read-only +
       SPECULATIVE-only + deterministic-replay invariants and the
       shared DiscoveryCandidateSink plumbing convergence (ADR-0080).

3. evals/CLAIMS.md — Tier 2 rows added

   - frontier_compare Lane A: determinism.primary_score, max_versor_condition
   - frontier_compare Lane B: prompt_battery.primary_score (CORE adapter),
     cross-provider artifact persistence
   - realizer_guard: all_claims_supported
   - contemplation: SPECULATIVE-only invariant, deterministic replay,
     additive sink path, no pack mutation (all CI-pinned by tests)

Verification
------------
$ core test --suite smoke -q
67 passed in 27.22s    (no regression)

$ uv run pytest -q tests/test_contemplation_loop.py \
    tests/test_contemplation_pipeline_convergence.py \
    tests/test_frontier_compare_cross_provider.py
27 passed in 4.87s

$ core bench --suite determinism --json 2>/dev/null | jq .results[0].passed
true        (was: JSONDecodeError on prior [pulse] pollution)

* feat(evals/ui): report viewer renders Lane B cross-provider + pass-rate chart

Stop-hook caught that #62 only covered contracts — the 929-line
report_viewer.html was never audited against the new cross-provider
report shape from #61.  Two real gaps:

1. Lane-aware observation drawer
   The drawer hardcoded Lane A (CORE-native) fields: surface,
   grounding_source, anchor_lens_mode_label, versor_condition.
   Lane B (cross-provider) observations carry different fields:
   provider, model, elapsed_ms, error_type, error_message.

   Loading a cross-provider report rendered only the surface row
   with empty `grounding` — the provider + model + timing data
   was unreachable without expanding "Show raw JSON".

   Fix: detect Lane B (presence of `obs.provider`) and render the
   appropriate field set.  Lane A still renders identically (now
   also surfaces trace_hash + register_id when present, which were
   silently buried in the raw JSON before).

2. Pass-rate chart per suite
   The summary strip showed one aggregate Primary % across all
   suites, with no way to see WHICH suite is dragging the score.
   Multi-suite runs (e.g. --suite all) had to expand each panel
   individually to find the failing one.

   Fix: new .passrate-chart element below the summary strip,
   one horizontal bar per suite showing passed/total.  All-pass =
   solid green, all-fail = solid red, partial = green/red split
   at the pass fraction.  CSS only — no new dependencies.

3. SUITE_PREAMBLES gains the prompt_battery entry so the sidebar
   shows the "side-by-side surface evidence across providers"
   description when loading a Lane B report.

Verified
--------
- Brace/paren/div balance unchanged (308/308 / 380/380 / 54/54)
- One <script> tag pair preserved
- Generated a real Lane B report via
  `python -m evals.frontier_compare --provider core --suite prompt_battery`
  for visual confirmation

Out of scope (noted for future PR)
----------------------------------
Sampled 3 `core demo` targets:
- register-tour: clean schema (all_claims_supported, claims, grid)
- audit-tour: both scene_1_* keys AND an empty scenes:[] array — inconsistent
- anti-regression: no all_claims_supported key, uses all_gates_held instead

Demo schema standardization deserves its own PR — operator tooling
would benefit from a uniform top-level success field across demos.

* docs(evals) + chore(demos): systematic audit + uniform success field

Stop-hook caught two real gaps after the contract+UI PR:
- demos had divergent success-field names (all_gates_held vs
  learning_loop_closed vs claim_supported vs nested claims_supported)
- no systematic look at the 48 eval directories had been done

Both addressed concretely; remaining work captured in audit doc
rather than vaguely deferred.

1. Demo schema standardization — uniform all_claims_supported field
----------------------------------------------------------------------
All 9 ``core demo`` targets now emit a top-level
``all_claims_supported: bool`` field.  Existing per-demo fields
(``all_gates_held``, ``learning_loop_closed``, ``claim_supported``,
nested ``claims_supported``) are preserved for backwards compat —
the new field is an alias derived from the demo's existing success
signal, not a replacement.

Operator tooling and the CI gate can now target
``all_claims_supported`` without knowing each demo's idiomatic
field name.

Files touched:
- evals/anti_regression/run_demo.py — adds AND of all_gates_held +
  active_corpus_byte_identical
- evals/learning_loop/run_demo.py — adds AND of learning_loop_closed +
  active_corpus_byte_identical
- scripts/publish_pack_measurements.py — adds AND of the three
  entries in the nested claims_supported dict
- evals/long_context_cost/comparison_runner.py — adds alias for
  claim_supported (singular)

The 5 demos already using ``all_claims_supported`` (audit-tour,
register-tour, anchor-lens-tour, orthogonality-tour, articulation)
are unchanged.

Verified across all 9 demos:
  audit-tour              : True
  register-tour           : True
  anchor-lens-tour        : True
  orthogonality-tour      : True
  pack-measurements       : True   ← new alias
  anti-regression         : True   ← new alias
  learning-loop           : True   ← new alias
  articulation            : True
  long-context-comparison : True   ← new alias

2. docs/EVAL_AUDIT_2026-05-20.md — systematic 48-lane audit
------------------------------------------------------------
Replaces the "future PR" deferral with a concrete document.

Contains:
- Method (what was inspected for each lane).
- Summary (40/48 have contract.md; 18/48 have saved results;
  empty results/ ≠ broken — most lanes regenerate on demand).
- Cross-provider relevance triage:
    * 9 lanes are cross-provider-relevant and could benefit
      from the prompt_battery-style adapter pattern (cognition,
      english_fluency_ood, hebrew_fluency, koine_greek_fluency,
      grammatical_coverage, inference_closure, multi_step_reasoning,
      discourse_paragraph, foundational_*_ood, etc.).
    * 29 lanes are CORE-only by design (versor closure, anchor
      lens, identity divergence, provenance, etc.) — wiring
      providers would be category-erroneous.
- Demo schema standardization status (this PR closes that).
- UI/UX coverage matrix.
- 5 concrete follow-up items, each focused enough for a single
  PR, none requiring architectural change.

Regenerated reports
-------------------
evals/long_context_cost/results/comparison_v1.json and
evals/results/phase2_pack_measurements.json now contain the new
all_claims_supported field (auto-regenerated when validating the
schema change).

evals/frontier_compare/results/sample_core_promptbattery.json
added as a reference Lane B report so the new viewer always has
something to load on first open.
2026-05-20 13:53:13 -07:00
Shay
db39a5aac7
chore(adr): rename ADR-0081 frontier provider adapters → ADR-0082 (#59)
Resolves a same-day numbering collision: the prior session produced
ADR-0080 + ADR-0081 (geometric stress field, falsified) in
docs/decisions/ while the frontier-provider-adapters work was
authored as ADR-0081 in a newly-created docs/adr/ directory,
unaware of the concurrent track.

This commit takes the minimum-blast-radius fix:
  - docs/adr/ADR-0081-...md → docs/adr/ADR-0082-...md
  - Update title header to ADR-0082, add "Renumbered from" breadcrumb
  - Update the two source-file docstrings that cite the ADR number
    (providers.py, model_registry.py)

The "two ADR directories" question (docs/adr/ vs docs/decisions/)
is NOT resolved here — docs/adr/ now has exactly one entry, while
docs/decisions/ is the canonical location per CLAUDE.md.  A future
PR should either consolidate or document the split; this commit
just unblocks the immediate naming collision.

Out of scope:
  - Consolidating directories
  - Renumbering anything in docs/decisions/
  - Re-numbering on the dev's local main (already pulled into this branch)
2026-05-20 12:46:13 -07:00
Shay
36904369ee feat(evals): ADR-0081 frontier provider adapters — .env.example, providers, model registry 2026-05-20 12:35:34 -07:00
Shay
06bbac86e1
feat(contemplation): ADR-0080 read-only speculative loop (#55)
* docs(adr): ADR-0080 contemplation loop boundary

* feat(contemplation): add read-only contemplation package

* feat(contemplation): add immutable speculative finding schema

* feat(contemplation): add deterministic substrate snapshot

* feat(contemplation): add frontier report miner package

* feat(contemplation): mine frontier compare failures as speculative findings

* feat(contemplation): add read-only contemplation runner

* feat(contemplation): add read-only contemplation CLI

* test(contemplation): prove read-only speculative loop invariants
2026-05-20 11:40:12 -07:00
Shay
9e791045eb docs(adr): ADR-0078 status Proposed → Ratified
Phase 1 telemetry shipped in 8e96728; bump the ADR status header to
match reality.
2026-05-20 06:30:48 -07:00
Shay
e550c4a240
docs(adr): ADR-0078 composer graph atom equivalence (#50) 2026-05-20 06:30:09 -07:00
Shay
8e96728009 feat(telemetry): ADR-0078 Phase 1 — composer/graph atom equivalence (observational)
Wires observational telemetry on the composer-vs-graph atom-set
relationship.  Phase 1 is strictly observational: no enforcement,
no surface mutation, no grounding-source change, no trace-hash impact.

New telemetry fields on TurnEvent + ChatResponse:
  composer_graph_atom_status         ∈ {equivalent, divergent,
                                         graph_unconstrained,
                                         composer_no_atoms,
                                         not_applicable, ""}
  composer_atom_set_hash             SHA-256 over sorted unique atoms
  graph_atom_set_hash                SHA-256 over sorted unique atoms
  composer_graph_atom_overlap_count  int

Composer atoms come from existing pack candidate metadata
(pack_semantic_domains channel through _maybe_pack_grounded_surface).
Graph atoms come from build_graph_from_input + resolve_lemma on
node.subject/predicate/obj — no prose parsing.  When a grounded
composer path lacks explicit atom provenance, status is
'composer_no_atoms'.

New pure helper:
  chat/atom_equivalence.py — normalize_atoms, hash_atoms,
  atoms_for_graph_nodes, compare_atom_sets

Tests (tests/test_composer_graph_atom_equivalence.py):
  - Pack DEFINITION path produces observable equivalence
  - Divergent atom sets produce distinct hashes
  - Register invariance: atom hashes + status identical across
    {neutral, terse, convivial}; trace_hash also constant (R5 axis)
  - Anchor lens engaged case still ASCII-only on surface
  - No prose-parsing helper symbols introduced in runtime.py
    (extract_candidate_surface_lemmas, surface_lemma,
    parse_surface_atoms) — enforces Phase 1 boundary

Performance note: build_graph_from_input now runs on every warm
English turn (previously only when forward_graph_constraint=True).
Phase 1 accepts this cost to make the telemetry universally
available; Phase 2+ can introduce a feature flag if needed.

Validation:
  - Cognition eval byte-identical: 100/100/91.7/100
  - Full lane: 2864 passed, 3 skipped, 0 failed (+5 over baseline)
  - Targeted lane: 72 passed in tests/test_{graph_constraint,
    pack_grounding,register_tour_demo,anchor_lens_tour_demo,
    orthogonality_tour_demo,realizer_guard_holdout,
    composer_graph_atom_equivalence}.py
2026-05-20 06:14:25 -07:00
Shay
5a78b0e37b feat(register): ADR-0077 — substantive register knobs + layering boundary (R6)
R5 (ADR-0072) shipped the register *machinery*; ADR-0074's orthogonality
tour proved the axis was decoratively orthogonal to anchor-lens but
inspection of the cognition-eval surfaces revealed two structural gaps:

* On pack-grounded DEFINITION/RECALL/COMPARISON composers, the only
  realizer override any register consumed was `disclosure_domain_count`
  — which only fires on the no-gloss disclosure path.  Under terse_v1,
  every gloss-DEFINITION cell was byte-identical to default_neutral_v1.
* The register-tour's `surfaces_vary_at_least_once` gate could be
  satisfied by convivial's decorative wrapper alone, masking that
  regression in CI.

R6 closes both:

Layering separation (the load-bearing fix):
* New TurnEvent/ChatResponse field `register_canonical_surface` carries
  the composer output BEFORE any register transformation.  The pipeline
  hashes this field for `trace_hash`, preserving R5's invariant that
  per-prompt trace_hash is CONSTANT across registers even while
  substantive transforms produce visibly different surfaces.

Substantive transforms (`chat/register_substantive.py`):
* terse_v1 gains 3 bool knobs: `drop_provenance_tag`, `compress_gloss`,
  `drop_articles` — all pure regex transforms on the canonical surface.
* convivial_v1 gains `append_semantic_domain_clause` — appends a single
  bounded "Related: <atom>." clause using the lemma's pack atoms.
* default_neutral_v1 leaves overrides empty; substantive transform is
  byte-identical no-op (preserves `byte_identity_null_lift`).
* C1 (ADR-0075) safety preserved: drop_articles refuses to drop
  articles following `not` (avoids R3 violations); no knob combination
  trips R2/R3.

Strengthened tour gate (`evals/register_tour/run_tour.py`):
* Replaces `surfaces_vary_at_least_once` with two falsifiable claims:
  - `terse_substantively_differs_from_neutral_on_pack_grounded_definition`
  - `convivial_substantively_differs_from_neutral_on_pack_grounded_definition`
  Both restrict to DEFINITION+pack-grounded cells and require
  difference beyond whitespace/punctuation.
* New claim `register_canonical_surfaces_identical` directly proves
  the layering separation.
* Preserves R5's `all_grounding_sources_identical` +
  `all_trace_hashes_identical`.

Pack ratification:
* Loader widened to accept `bool` for closed-set R6 keys
  (drop_provenance_tag / compress_gloss / drop_articles /
  append_semantic_domain_clause).
* `_KNOWN_OVERRIDE_KEYS` ratify gate extended with same.
* terse_v1 + convivial_v1 reratified with new knobs; companion
  mastery reports re-sealed.  default_neutral_v1 unchanged.

Invariants pinned:
* `invariant_register_canonical_surface_constant_across_registers` (new)
* `invariant_terse_substantively_distinct_from_neutral` (new)
* `invariant_convivial_substantively_distinct_from_neutral` (new)
* `invariant_realizer_no_illegal_articulation` (C1, preserved)
* `invariant_realizer_guard_byte_identity_on_currently_passing_cases`
  (C1, preserved)

Verification:
* `core eval cognition`: 100.0% / 91.7% / 100.0% / 100.0% — byte-
  identical under default_neutral_v1.
* `core demo register-tour`: all 5 claims green, exit 0.
* `core demo anchor-lens-tour`: green (no anchor-lens code touched).
* `core demo orthogonality-tour`: green (5/5 claims).
* Full lane: 2858 passed, 1 pre-existing failure
  (test_all_preamble_explains_combined_run, carried forward
  unchanged from main).  56 new R6 tests across three files.
2026-05-19 23:39:11 -07:00
Shay
d7499c80b3
feat(intent): normalize confirmation-tag propositions (#45) 2026-05-19 22:55:28 -07:00
Shay
7cc2888ed2 feat(coherence): ADR-0075 — realizer slot-type guard (C1)
C1 coherence floor: a deterministic verifier that runs on every
candidate surface produced by the truth path, before assignment to
ChatResponse.surface.  Rejects illegal articulations and routes them
to a bounded disclosure string — admission control with a
deterministic fallback, not normalization.

Active rules (R1 deferred during ratification — see ADR):
  R2_aux_neg_requires_verb     — "<aux> not <wrong-POS>"  rejected
  R3_be_neg_requires_predicate — "<be>  not <verb>"       rejected

Fail-open on unknown POS, fail-closed on explicit wrong POS.
Cognition eval byte-identical (100/91.7/100/100).

Original bug class — "Light reveals truth, right?" → "Right does not
thought." — now routes to "I do not have a reviewed articulation for
that yet." with grounding_source=none, walk_surface preserving the
rejected candidate, and telemetry carrying R2_aux_neg_requires_verb.

Files:
  generate/realizer_guard.py            NEW — pure verifier
  chat/runtime.py                       hook on stub + main paths
  chat/telemetry.py                     serialize guard fields
  core/physics/identity.py              TurnEvent +2 fields
  evals/realizer_guard/run_holdout.py   NEW — 6-prompt cluster
  tests/test_realizer_guard_*.py        NEW — 46 tests (unit/seam/holdout)
  docs/decisions/ADR-0075-*.md          NEW — ratified

Invariants pinned:
  invariant_realizer_no_illegal_articulation
  invariant_realizer_guard_byte_identity_on_currently_passing_cases

Lanes (excluding 1 pre-existing TestDemoPreambles failure unrelated
to C1, already present at 4426f38):
  smoke 67/67  cognition 120/120(+1s)  teaching 17/17
  packs 6/6   runtime 19/19   algebra 132/132   full 2792/2793
2026-05-19 22:35:09 -07:00
Shay
4426f387d1 feat(demo): ADR-0074 — orthogonality tour (anchor-lens × register)
A single demo that walks the full 3 × 3 × 2 matrix (register × lens
× prompts, 18 cells) and pins five claims simultaneously, packaging
both single-axis invariants into one composition gate.

The single-axis tours assert opposite invariants:

  register-tour    : per (lens, prompt), trace_hash CONSTANT across
                     registers (R5 / ADR-0072).
  anchor-lens-tour : per (register, prompt), engaged lens diverges
                     in trace_hash from the unanchored baseline
                     (L1.4 / ADR-0073d).

Orthogonality-tour packages both claims simultaneously across the
full matrix, plus three surface-level claims that pin the markers
operators actually see.

Composed claims (all five must hold)

  A) inner_register_invariant_within_lens
     For each (lens, prompt) cell, the three register runs share an
     identical trace_hash.  (R5 register-tour, applied 6 times:
     3 lenses × 2 prompts.)

  B) outer_lens_distinctness_within_register
     For each (register, prompt) cell where any non-unanchored lens
     engages, that engaged lens's trace_hash differs from the
     unanchored baseline at the same (register, prompt).
     (L1.4 anchor-lens-tour, applied 6 times: 3 registers × 2 prompts.)

  C) surface_carries_register_marker_under_convivial
     Every convivial cell with a non-empty surface has a non-empty
     register_variant_id.

  D) surface_carries_lens_annotation_when_engaged
     Every engaged cell carries [lens(<id>):<mode>] in surface AND
     a non-empty anchor_lens_mode_label.

  E) no_substrate_glyph_leak_across_grid
     No cell's surface contains Greek/Hebrew/Syriac/Arabic glyphs.
     (ADR-0073c gate re-asserted across the full matrix.)

CLI wiring

  core demo orthogonality-tour            human-readable grid + claims
  core demo orthogonality-tour --json     structured report

Exit code 0 iff all five claims hold.

Files

  evals/orthogonality_tour/__init__.py             NEW
  evals/orthogonality_tour/run_tour.py             NEW
  core/cli.py                                       EDIT
    - cmd_demo handler wires orthogonality-tour
    - demo choices + EPILOG examples updated
  tests/test_orthogonality_tour_demo.py             NEW (9 tests)
  docs/decisions/ADR-0074-orthogonality-tour.md     NEW

Sanity check baked into tests
  test_engaged_cells_appear_for_both_non_trivial_lenses pins that
  grc_logos_v1 engages on knowledge in all 3 registers (3 cells)
  and he_logos_v1 engages on truth in all 3 registers (3 cells).
  Prevents the lift claims being vacuously satisfied by a future
  engagement regression.

Lane evidence

  - 9 new orthogonality-tour tests pass.
  - core demo register-tour      → all_claims_supported: True
  - core demo anchor-lens-tour   → all_claims_supported: True
  - core demo orthogonality-tour → all_claims_supported: True
  - python -m core.cli eval cognition → byte-identical 100/100/91.7/100.
  - Full lane: 2745 passed / 4 skipped / 1 pre-existing failure
    (+9 over L1.4's 2736; the one failure remains
    test_all_preamble_explains_combined_run, unrelated).

No runtime / composer / loader / pack / schema changes.  Pure demo
consumer of existing telemetry contracts.
2026-05-19 20:33:33 -07:00
Shay
1feec74b1c feat(anchor_lens): ADR-0073d — L1.4 telemetry, CLI flag, tour demo
L1.4 closes the anchor-lens inside-out arc (L1.1→L1.4 mirroring
R1→R5).  Substantive axis is now operator-observable,
operator-driven, and demo-falsifiable — exactly what R5 did for
the register subsystem.

Telemetry extension
  - TurnEvent + ChatResponse gain anchor_lens_id +
    anchor_lens_mode_label (both default "" → pre-L1.4
    byte-identical).
  - serialize_turn_event surfaces both fields in every JSONL line.
  - Mode-label extracted via _ANCHOR_LENS_ANNOTATION_RE from the
    PRE-decoration surface (so register decoration cannot interfere
    with anchor-lens telemetry).  Composer remains the sole source
    of truth for engagement; the runtime helper is read-only.

Operator surface
  - core chat --anchor-lens <id> CLI flag threads into
    RuntimeConfig.anchor_lens_id.
  - Invalid id → AnchorLensError caught at cmd_chat and surfaced
    as _die("invalid --anchor-lens pack id: ...", code=2) before
    the REPL launches.
  - Composes with --register (both flags wire through
    _runtime_config_from_args).

Narrative demo
  - evals/anchor_lens_tour/run_tour.py walks 2 prompts × 3
    ratified lenses ({default_unanchored_v1, grc_logos_v1,
    he_logos_v1}).  Asserts four claims:
      * lens_ids_recorded_per_turn
      * trace_hashes_distinct_across_lenses (OPPOSITE of
        register-tour's identical-hash claim)
      * surface_propositions_distinct_across_lenses
      * no_substrate_glyph_leak (block-scoped Greek/Hebrew/
        Syriac/Arabic; stylistic punct allowed)
  - Exit code 0 iff all four hold.
  - Bundled into `core demo` choices + EPILOG.

Tests (30 new)
  - tests/test_anchor_lens_telemetry.py (16) — TurnEvent shape,
    serializer keys, runtime emits per lens / per engagement
    state, ChatResponse mirrors event, mode-label extractor unit.
  - tests/test_anchor_lens_cli.py (9) — _runtime_config_from_args
    threading, invalid id fail-fast, parser flag wiring, parser
    composes with --register.
  - tests/test_anchor_lens_tour_demo.py (9) — four seam claims
    pinned individually + all_claims_supported + per-cell
    anchor_lens_id + unanchored cells empty mode + engaged cells
    carry mode label.

Lane evidence
  - 30 new L1.4 tests pass.
  - core demo anchor-lens-tour --json → all_claims_supported: True.
  - core demo register-tour --json    → all_claims_supported: True.
    Both tours pass simultaneously — orthogonality CI-pinned.
  - python -m core.cli eval cognition → public 100/100/91.7/100
    byte-identical (lens=None / default_unanchored_v1).
  - Full lane: 2736 passed / 4 skipped / 1 pre-existing failure
    (+30 over L1.3's 2706; the one failure remains
    test_all_preamble_explains_combined_run, unrelated).

Live demo (canonical proof)
  P1: 'What is knowledge?'
    default_unanchored_v1  trace=17c9aabe…  mode=(none)
    grc_logos_v1           trace=0198ad4c…  mode=systematic
    he_logos_v1            trace=17c9aabe…  mode=(none)
  P2: 'What is truth?'
    default_unanchored_v1  trace=2557f3e8…  mode=(none)
    grc_logos_v1           trace=2557f3e8…  mode=(none)
    he_logos_v1            trace=ec8d84aa…  mode=covenant-verity

  Engagement is substrate-scoped: grc never touches truth, he
  never touches knowledge.  Trace hashes diverge exactly where the
  lens engages.

Trust boundaries
  - --anchor-lens flag does not bypass ratification; loader still
    enforces companion mastery report self-seal + ratify-time
    substrate-atom existence check (ADR-0073b/c).
  - Mode-label extraction is read-only regex parse; can't forge
    annotations the composer didn't emit.
  - Telemetry stays redact-safe — both fields are identifiers /
    mode labels, not content.  include_content=False emits them
    unconditionally.
  - No new mutation surface; pack files unchanged.

Closes the anchor-lens inside-out arc
  L1.1  content prerequisite                  ✓ (ADR-0073a)
  L1.2  class + loader + unanchored sentinel  ✓ (ADR-0073b)
  L1.3  first lenses + composer wiring        ✓ (ADR-0073c)
  L1.4  telemetry + CLI + tour demo           ✓ (this commit)

  Mirrors the R1→R5 register cadence exactly.  Both axes are now
  operator-observable, CI-falsifiable, audit-traceable, and
  composable via the orthogonality claim pinned in both tours.
2026-05-19 20:21:41 -07:00
Shay
b35bec6465 feat(anchor_lens): ADR-0073c — L1.3 first lenses + composer wiring
L1.3 of the anchor-lens inside-out rollout — first substantive
surface lift on the substantive axis.  Two ratified non-trivial
lenses engage on cognition-pack lemmas via the alignment graph,
appending [lens(<id>):<mode>] annotations to the existing
pack-grounded surface.

Two ratified lenses

  grc_logos_v1 (Greek substrate)
    primary_substrate         : "grc"
    semantic_domain_preferences: ["logos.episteme.systematic_knowledge"]
    cognitive_mode_label       : "systematic"
    Engages on en "knowledge" via grc-core-cog-021 (ἐπιστήμη) →
    en-core-cog-007 alignment edge.

  he_logos_v1 (Hebrew substrate)
    primary_substrate         : "he"
    semantic_domain_preferences: ["logos.aletheia.verity"]
    cognitive_mode_label       : "covenant-verity"
    Engages on en "truth" via he-core-cog-002 (אמת) →
    en-core-cog-002 alignment edge.

  Both ratified under method anchor_lens_lifts_proposition.

Engagement rule (single)

  1. Resolve en_lemma → entry_id (cognition pack).
  2. For each substrate pack matching lens.primary_substrate, load
     alignment.jsonl; find edges where target_id == entry_id.
  3. For each such substrate lemma, if any atom in its
     semantic_domains ∈ lens.semantic_domain_preferences → engage.
  4. No match → None (no annotation; byte-identical surface).

The pivot is shared semantic_domain atoms surfaced via the
alignment graph — exactly the language-neutral commitment from
ADR-0073.  Engagement never touches non-English surface text;
entry_ids and atom strings only.

Surface lift

  no-lens : "Knowledge is X. pack-grounded (en_core_cognition_v1)."
  lens-on : "Knowledge is X. pack-grounded (en_core_cognition_v1) [lens(grc_logos_v1):systematic]."

  Annotation between existing provenance and trailing period.
  Both metadata fields are ASCII-bounded ≤64 chars at the loader
  level, so the annotation can never carry non-ASCII.

Scope deliberately narrow

  L1.3 wiring restricted to pack_grounded_surface /
  build_pack_surface_candidate (DEFINITION/RECALL only).  Other
  composers (COMPARISON / CORRECTION / PROCEDURE / NARRATIVE /
  EXAMPLE / CAUSE / VERIFICATION) accept the anchor_lens kwarg via
  forward-compat default UNANCHORED but do not yet consume it.
  L1.3b or later broadens to those intent shapes.

Ratify gate widening

  Non-null lenses must:
    - have primary_substrate ∈ {grc, he, en}
    - have a non-empty cognitive_mode_label
    - every preferred atom must exist in at least one lemma of the
      named substrate (trust boundary: operators cannot ship a lens
      pointing at atoms not on disk).
  Method: anchor_lens_lifts_proposition.  Null lenses still ratify
  under byte_identity_null_lift (L1.2 method).

Seam allow-list widening

  Truth-path modules (cognition / trace / pipeline / intent /
  propagation / vault / algebra) still refused.  Composer-side
  imports from chat/pack_grounding.py now permitted — the same way
  ADR-0069's R2 widened the register seam.

New invariants pinned (3)

  tests/test_anchor_lens_engagement_unit.py (14 tests) — resolver
  returns mode label only on intended substrate × en lemma pair;
  case-insensitive; engagement None under null lens; synthetic
  lens with unmatched atom returns None; annotation is pure ASCII.

  tests/test_anchor_lens_lifts_proposition.py (17 tests) — grc
  engages on knowledge only, he engages on truth only,
  cross-lens isolation, three-way distinctness, replay determinism
  per (lens × prompt), register-tour seam holds within each lens
  scope (orthogonality CI-pinned, parametrized over 4 lens
  choices).

  tests/test_anchor_lens_no_glyph_leak.py (5 tests) — hard
  block-scoped gate: Greek (U+0370..03FF, U+1F00..1FFF), Hebrew
  (U+0590..05FF), Syriac, Arabic.  Stylistic punctuation
  (em-dash etc.) explicitly allowed; em-dash predates L1.3 by a
  wide margin and is not a substrate-leak risk.  Tested per-lens
  across every cognition case + direct lens-metadata ASCII check.

Lane evidence

  74 anchor-lens tests pass (37 from L1.2 + 37 new).
  python -m core.cli eval cognition → public 100/100/91.7/100
  byte-identical (lens=None / default_unanchored_v1).
  core demo register-tour --json → all_claims_supported: True
  (R5 seam still holds; L1.3 doesn't perturb presentation axis).
  Full lane: 2706 passed / 4 skipped / 1 pre-existing failure
  (+37 over L1.2's 2669; the one failure remains
  test_all_preamble_explains_combined_run, unrelated).

Files

  packs/anchor_lens/grc_logos_v1.json                        NEW
  packs/anchor_lens/grc_logos_v1.mastery_report.json         NEW
  packs/anchor_lens/he_logos_v1.json                         NEW
  packs/anchor_lens/he_logos_v1.mastery_report.json          NEW

  scripts/ratify_anchor_lens_packs.py                        EDIT
    LENS_IDS adds grc_logos_v1 / he_logos_v1; gate widened.

  chat/pack_grounding.py                                     EDIT
    _resolve_anchor_lens_mode, _maybe_append_anchor_lens_annotation,
    _substrate_lexicon_by_entry_id, _en_lemma_to_entry_id.
    build_pack_surface_candidate + pack_grounded_surface gain
    anchor_lens kwarg (default UNANCHORED).

  chat/runtime.py                                            EDIT
    Thread self.anchor_lens into pack_grounded_surface() call.

  tests/test_anchor_lens_pack_seam.py                        EDIT
    Doc-comment updated for L1.3 allow-list.

  tests/test_anchor_lens_*                                   NEW (3 files)

  docs/decisions/ADR-0073c-anchor-lens-composer-wiring.md    NEW
2026-05-19 20:06:02 -07:00
Shay
9b1b63b253 feat(anchor_lens): ADR-0073b — L1.2 class + loader + unanchored sentinel
L1.2 of the anchor-lens inside-out rollout — pack class, loader,
ratified sentinel pack, and runtime threading.  Mirrors the
ADR-0068 register-class pattern exactly.  No composer consumes the
lens yet — that's L1.3.

AnchorLens frozen dataclass (packs/anchor_lens/loader.py)
  - lens_id / version / description / display_name
  - primary_substrate ∈ {grc, he, en, none}
  - semantic_domain_preferences: tuple[str, ...] (ordered, ≤64 atoms
    of ≤64 chars each, no duplicates)
  - cognitive_mode_label: str (≤64 chars)
  - mastery_report_sha256
  - is_unanchored() / is_null_lens() predicates
  - unanchored() classmethod + module-level UNANCHORED singleton

Loader contract (mirror of packs/register/loader.py)
  - safe_pack_id path-traversal rejection
  - Schema validation + envelope bounds checks
  - Companion mastery report self-seal + report_sha256 verification
  - CORE_ALLOW_UNRATIFIED_ANCHOR_LENS=1 dev bypass
  - require_ratified default True
  - No truth-path imports (pinned by seam test)

default_unanchored_v1 ratified pack
  - Null lens: primary_substrate="none", empty preferences,
    empty cognitive_mode_label
  - Self-sealed at b3235072fdbb2219...
  - Ratification method: byte_identity_null_lift
  - scripts/ratify_anchor_lens_packs.py L1.2 gate accepts only
    null lenses; L1.3 will widen.  Idempotent.

RuntimeConfig threading
  - new field: anchor_lens_id: str | None = None
  - new constant: DEFAULT_ANCHOR_LENS = "default_unanchored_v1"
  - ChatRuntime.__init__ loads the lens (None → AnchorLens.
    unanchored(); otherwise load_anchor_lens(id)) and stores as
    self.anchor_lens + self.anchor_lens_id.  Invalid ids fail-fast
    at init via AnchorLensError, not at first turn.
  - No composer reads the attribute yet.

Tests pinned (37 total)
  - tests/test_anchor_lens_pack_loader.py (24) — load happy path,
    sentinel structural identity, invalid id rejection (traversal,
    empty, slashes, missing), ratification bypass paths, companion
    SHA mismatch, bounds (substrate / preferences / atoms / label /
    duplicates / capacity), field-missing, lens_id mismatch with
    filename, unsupported schema_version.
  - tests/test_anchor_lens_null_lift.py (4) — load-bearing L1.2
    invariant `anchor_lens_byte_identity_null_lift`: full public
    cognition lane byte-identical for surface, trace_hash, and
    aggregate metrics between anchor_lens_id=None and
    "default_unanchored_v1".
  - tests/test_anchor_lens_pack_seam.py (9) — AST refuses any
    `packs.anchor_lens` import from truth-path modules (cognition /
    trace / pipeline / intent / propagation / vault / algebra) AND
    refuses any truth-path import from the loader itself.

Lane evidence
  - All 37 anchor-lens tests pass.
  - python -m core.cli eval cognition → public 100/100/91.7/100
    byte-identical (lens loaded but no composer reads it).
  - core demo register-tour --json → all_claims_supported: True
    (R5 seam still holds; L1.2 doesn't perturb register).
  - Full lane: 2669 passed / 4 skipped / 1 pre-existing failure
    (+37 over L1.1's 2632; the one failure remains
    test_all_preamble_explains_combined_run, unrelated).

Trust boundaries (per CLAUDE.md / ADR-0051)
  - safe_pack_id path-traversal rejection at loader entry.
  - No dynamic imports.
  - Loader is read-only; mutation only via ratify script.
  - Seam test refuses any new anchor-lens import upstream of the
    realizer.  L1.3 will widen the allow-list to include composer
    files at the same time it adds composer behaviour — exactly the
    way the register seam was widened at R2.

What L1.2 deliberately does NOT do
  - No composer consumes the lens (that's L1.3).
  - No TurnEvent / ChatResponse telemetry fields (L1.4).
  - No `core chat --anchor-lens` CLI flag (L1.4).
  - No anchor-lens-tour demo (L1.4).
2026-05-19 19:46:34 -07:00
Shay
2dd50b8dc4 feat(packs): ADR-0073a — anchor lens L1.1 content phase
Umbrella ADR-0073 ratified (Accepted); L1.1 content phase
(ADR-0073a) landed.  Pure pack enrichment — no runtime code, no
composer change, no test of behaviour.  Substrate prerequisite for
the L1.2–L1.4 phases.

Greek additions (grc_logos_cognition_v1, 20 → 29 entries)
  Knowledge family (English collapses to `knowledge`):
    - ἐπιστήμη  logos.episteme.systematic_knowledge
    - σύνεσις   logos.synesis.insight
    (γνῶσις at grc-core-cog-007 unchanged — treated as the
     experiential variant by the L1.3 lens config)
  Love family (English collapses to `love`):
    - ἀγάπη   logos.agape.covenant_love
    - φιλία   logos.philia.companion_love
    - ἔρως    logos.eros.passionate_love
    - στοργή  logos.storge.familial_love
  Time family (English collapses to `time`):
    - αἰών    logos.aion.age_era
    - χρόνος  logos.chronos.clock_time
    - καιρός  logos.kairos.opportune_moment

Hebrew additions (he_core_cognition_v1, 20 → 23 entries)
  - חסד    logos.chesed.covenant_loyalty
  - שלום   logos.shalom.wholeness_peace
  - צδק    logos.tzedek.right_order

Alignment.jsonl on both cognition-tier packs (previously only the
micro packs carried alignment)
  - grc_logos_cognition_v1/alignment.jsonl — 20 edges: three-way core
    dyads (word / truth / light / life / beginning / wisdom),
    knowledge-family → en collapse, ἀγάπη↔חסד covenant-love pairing
    (weight 0.86, Septuagintal), `cross_lang.no_english_collapse`
    annotations for love + time families pointing at
    `en-collapse-<family>` sentinel ids (weight 0.0).
  - he_core_cognition_v1/alignment.jsonl — 7 edges: core dyads to en,
    חסד↔ἀγάπη covenant pairing, no-english-collapse annotations for
    חסד / שלום / צδק.

Manifest checksums refreshed per CLAUDE.md doctrine
  - grc_logos_cognition_v1: b45bcf581cee… → 0f9436675707…
  - he_core_cognition_v1:   dee1e8c6ad9a… → 22145d008185…

Design decisions
  - Existing 20 + 20 lemma atoms untouched — downstream tests /
    composers / teaching chains keep referencing the same atoms.
    Only new lemmas carry the distinguishing atoms.
  - `cross_lang.no_english_collapse` edges are metadata not data
    (sentinel target ids, weight 0.0).  Their purpose is letting the
    alignment graph answer "does English split this family?" without
    forcing an artificial English lemma.
  - Every new entry carries `adr-0073a:hand_authored:2026-05-19` in
    its `provenance_ids` so future audits can find the L1.1 cohort
    deterministically.

Verification
  - python -m language_packs verify grc_logos_cognition_v1   → OK
  - python -m language_packs verify he_core_cognition_v1     → OK
  - python -m language_packs compile <both>                  → 29 / 23
    manifold points; spot-check confirms καιρός / צδק resolve.
  - python -m core.cli eval cognition                        → public
    100 / 100 / 91.7 / 100 byte-identical (new lemmas sit on disk but
    no composer references them yet).
  - python -m core.cli test --suite cognition                → 120/1 pass
  - python -m core.cli test --suite smoke                    → 67/0 pass
  - python -m core.cli test --suite full                     → 2632 passed
    / 4 skipped / 1 pre-existing failure (test_all_preamble_explains_
    combined_run rename drift, unrelated).
  - core demo register-tour                                  → exit 0
    (R5 seam still holds; L1.1 doesn't touch register pathway).

What L1.1 deliberately does NOT do
  - No AnchorLens class (that's L1.2 / ADR-0073b).
  - No composer wiring (L1.3 / ADR-0073c).
  - No --anchor-lens CLI flag or demo (L1.4 / ADR-0073d).
  - No teaching corpus in non-English (post-L1).
2026-05-19 19:30:20 -07:00
Shay
f673c0eb06 docs(adr): ADR-0073 — anchor lens substrate (Proposed)
Umbrella ADR for the substantive-variation axis that composes
orthogonally against register (ADR-0068..0072).  Drafted only;
status Proposed.  No code, no pack, no test landed.

Architecture summary
  - Anchor lens is the substantive axis: register varies surface text
    while keeping grounding_source / trace_hash byte-identical;
    anchor lens deliberately moves both because the proposition
    itself changes when the substrate changes.
  - Pivot is shared `semantic_domains` atoms (already on disk across
    grc / he / en cognition packs), not transliteration tables — the
    seam stays language-neutral so future substrates compose without
    touching anchor-lens code.
  - English compound phrasing only at the surface ("knowing-as-
    experience", "knowing-as-system"); Greek / Hebrew glyphs live in
    audit / provenance fields only.  L1.3 invariant
    `anchor_lens_no_glyph_leak` is a hard gate.

Four-phase rollout (mirrors R1–R5 cadence)
  L1.1  content phase — distinction-bearing lemma additions
        (ἐπιστήμη / σύνεσις / ἀγάπη-φιλία-ἔρως-στοργή / αἰών-χρόνος-
        καιρός; חסד / שלום / צδק) + alignment.jsonl on the cognition-
        tier packs.  No code.  Prerequisite for every later phase.
  L1.2  AnchorLens pack class + loader + `default_unanchored_v1`
        sentinel.  Null-lift CI invariant pinned.
  L1.3  First non-trivial lenses (`grc_logos_v1`, `he_logos_v1`)
        wired into chat/pack_grounding.py composers.  Proposition-
        lift invariant + glyph-leak gate pinned.
  L1.4  Telemetry (TurnEvent + ChatResponse gain anchor_lens_id),
        `core chat --anchor-lens` flag, `core demo anchor-lens-tour`
        asserting trace_hashes_distinct_across_lenses (opposite of
        register-tour's claim — both must hold).

Three honest gaps blocking L1.2+
  - Distinction-bearing lemmas absent from cognition packs.
  - No reviewed teaching corpus for non-English (cognition_chains,
    relations_chains, cross_pack_chains all en-only).
  - No realizer infrastructure for cross-lingual surface composition.

L1.1 (pure content) closes all three for the cognition tier.

Orthogonality claim — load-bearing
  register-tour    : per prompt, fix lens, vary register → trace_hash CONSTANT
  anchor-lens-tour : per prompt, fix register, vary lens → trace_hash DISTINCT
  Both must continue to hold; failure of either breaks the seam.
2026-05-19 19:13:01 -07:00
Shay
7f0bad3e20 feat(register): R5 — operator-visible register telemetry + tour demo
ADR-0072 ratified + implemented.  Closes the register subsystem
inside-out arc (R1 ADR-0068 → R5 ADR-0072): the presentation axis is
now operator-visible, CI-falsifiable, and audit-traceable.

Telemetry extension
  - TurnEvent + ChatResponse gain register_id + register_variant_id
    (12-char SHA-256 prefix of selected (opening, closing) pair;
    empty string for UNREGISTERED / no-decoration registers).
  - serialize_turn_event surfaces both fields in every audit JSONL
    line.  Pre-R5 callers stay byte-identical (defaults are "").

Decoration result widened
  - chat/register_variation.py: decorate_surface now returns
    DecorationResult(surface, opening, closing, variant_id).
  - decorate_surface_str alias preserves the pre-R5 string-only API
    for off-runtime callers.
  - chat/runtime.py updated at both call sites (stub + main).

Operator surface
  - core chat --register REGISTER_ID threads into
    RuntimeConfig.register_pack_id via _runtime_config_from_args.
  - Invalid id ⇒ RegisterPackError caught at cmd_chat and surfaced
    as a clean _die(...) before the REPL launches.

Narrative demo
  - evals/register_tour/run_tour.py walks 4 prompts × 3 ratified
    registers ({default_neutral_v1, terse_v1, convivial_v1}) and
    asserts three load-bearing seam claims:
      * all_grounding_sources_identical
      * all_trace_hashes_identical (ADR-0069 invariant C, falsifiable)
      * surfaces_vary_at_least_once (ADR-0071 seeded variation lift)
  - core demo register-tour exit code = 0 iff every claim holds.

Tests
  - tests/test_register_telemetry.py (6) — TurnEvent default,
    serializer keys, runtime emits register_id/variant_id for
    convivial/terse/unregistered, ChatResponse mirrors event fields.
  - tests/test_register_cli.py (7) — _runtime_config_from_args
    threading, invalid-id fail-fast, parser wires --register.
  - tests/test_register_tour_demo.py (7) — three seam claims pinned
    individually + all_claims_supported + per-cell register_id +
    variant_id discipline (empty for neutral/terse, non-empty for
    convivial).
  - tests/test_register_variation.py extended (4 new) — DecorationResult
    shape, decorate_surface_str alias, variant_id stability,
    bijection between non-trivial marker pairs and variant_ids.

Lane evidence
  - Full lane: 2632 passed / 4 skipped / 1 pre-existing failure
    (tests/test_cli_demo.py::test_all_preamble_explains_combined_run,
    unrelated to R5).
  - Cognition eval byte-identical: public 100 / 100 / 91.7 / 100.

Trust boundaries (per CLAUDE.md)
  - --register flag does not bypass ratification; loader validates the
    pack id through _find_pack and the ratify gate at load time.
  - variant_id is content-addressed; no raw markers leak into audit.
  - Telemetry stays redact-safe — register_id and variant_id are
    identifiers, not content, so include_content=False emits them
    unconditionally.
  - No new mutation surface; pack files on disk are not modified.
2026-05-19 19:03:07 -07:00
Shay
6207b5fd0e feat(register): R1–R4 register pack subsystem — deterministic surface variation
Introduces the presentation axis as a fourth pack class (sibling to identity /
safety / ethics), orthogonal to the truth path. Same input + same packs +
same register ⇒ bit-for-bit reproducible surface; varying any of the three ⇒
genuinely different output. No stochastic sampling.

ADR-0068 (R1): RegisterPack frozen dataclass, loader, ratify script, seam test.
  - default_neutral_v1 ratified as null register.

ADR-0069 (R2): realizer register parameter threaded through 9 composer entry
  points; RuntimeConfig.register_pack_id; three byte-identity invariants
  (A: None ≡ pre-R2 unregistered; B: None ≡ default_neutral_v1; C: trace_hash
  invariant under register). Amended to default-with-lint after 167-call-site
  scout: composers default to UNREGISTERED, AST lint enforces explicit
  register= at runtime call sites.

ADR-0070 (R3): terse_v1 register, first non-neutral pack. realizer_overrides
  schema with known-keys allow-list (disclosure_domain_count ∈ {1,2,3}).
  build_pack_surface_candidate reads override with fail-soft clamp. New
  invariant register_invariant_grounding asserts grounding_source +
  trace_hash byte-identical across {None, neutral, terse}.

ADR-0071 (R4): seeded surface variation via convivial_v1.
  chat/register_variation.py applies SHA-256-seeded marker selection from
  bounded discourse-marker buckets. ChatResponse.pre_decoration_surface routes
  truth-path surface to core/cognition/pipeline.py so trace_hash stays
  invariant under register (the load-bearing architectural fix — initially
  invariant C failed under convivial because decoration was leaking into
  trace_hash via response.surface). Empty-string marker entries now
  legitimate ("no marker this turn" is a valid seed pick). realizer_overrides
  schema widened with per_intent nested block (validated against IntentTag
  whitelist; wired but not exercised by convivial). Two new invariants:
  seeded_variation_replay_equivalence (fresh runtimes → byte-identical) and
  seeded_variation_turn_distinct (same prompt across turns → ≥2 distinct
  surfaces).

ADR-0072 (R5, draft): telemetry + operator surface — TurnEvent gains
  register_id and register_variant_id, core chat --register flag, core demo
  register-tour. Status: Proposed; not yet implemented.

Three ratified register packs ship: default_neutral_v1 (null), terse_v1
(disclosure_domain_count=1), convivial_v1 (3 openings × 3 closings).

Verification:
  - 84 register tests pass + 1 documented skip
  - Curated lanes green: smoke 67, cognition 120+1s, teaching 17, packs 6,
    runtime 19, algebra 132
  - Cognition eval byte-identical to pre-register baseline:
    public 100/100/91.7/100, holdout 100/100/83.3/100
  - Full lane: 2608 passed, 4 skipped, 1 failed (pre-existing
    test_cli_demo.py "Combined Demo" → "Run Every Demo" rename, unrelated)

Truth-path isolation: chat/register_variation.py is realizer-side; the seam
test (tests/test_register_pack_seam.py) refuses imports of packs.register
from intent classification, propagation, vault recall, trace hashing, and
algebra.
2026-05-19 16:52:36 -07:00
Shay
90fc1b40a0 docs(evals): articulation benchmark preamble — discourse-planner spine
Records the deterministic, grounded, multi-clause articulation
benchmark that the discourse-planner work has stabilised.  Mirrors
the format of teaching_loop_bench.md so the four sub-benches in
benchmarks/articulation.py have a load-bearing reference document.

Headline:

* 20 independent ChatRuntime instances × 4 prompts (EXPLAIN /
  PARAGRAPH / COMPOUND / WALKTHROUGH) produce 4 unique surfaces —
  byte-identical determinism on the articulation path with
  RuntimeConfig(discourse_planner=True).
* Every visible token traces to a pack lemma, pack gloss, reviewed
  teaching-chain entry, or fixed-template connective from the
  closed five-entry _MOVE_CONNECTIVE table.  No synthesis.
* discourse_planner sub-bench:
    cases:                     4
    articulate_sentence_rate:  1.0
    disclosure_sentence_rate:  0.0
    multi_sentence_rate:       1.0
* Compound prompt ("What is truth, and why does it matter?") emits
  6 distinct grounded sentences with cross-part fact dedup, no
  anchor repetition.
* Walkthrough mode walks the teaching-chain edge graph up to 3 hops,
  cycle-safe, final hop as CLOSURE; no chain ⇒ degrades to ANCHOR +
  SUPPORT rather than fabricating steps.

Doc explains the partitioned predicate contract
(articulate + disclosure + unarticulate = 1.0, total and disjoint)
so future readers know why ``multi_sentence_rate`` alone is not the
headline.

Companion docs cross-linked: discourse_runtime_baseline_2026-05-19.md
(lane-level delta table), the two new isolation lanes
(compound_intent_decomposition, walkthrough_chain), and the
partitioned multi_sentence_response contract.
2026-05-19 12:47:38 -07:00
Shay
6dd8efe7b3 feat(intent): expository-DEFINITION rules for Explain/Paragraph prompts
Extends ``generate/intent.py:_RULES`` with three new expository
patterns so the upstream subject-extraction gap that the dedup
revealed is closed:

* ``^explain\s+``                                  → DEFINITION
* ``^(write|compose|draft) (a )?(short|brief)?
   paragraph (about|on)\s+``                       → DEFINITION
* ``^paragraph (about|on)\s+``                     → DEFINITION

Rules placed AFTER the NARRATIVE family so ``Tell me about X`` and
``Describe X`` continue to route to NARRATIVE.  Subject extraction
re-uses ``_normalize_subject`` so articles and trailing punctuation
are stripped: ``Explain the parent.`` → subject ``parent``.

``ResponseMode`` is untouched and remains orthogonal: the same prompts
still classify as ``EXPLAIN`` / ``PARAGRAPH`` independently.

20 new tests pin: each rule's expected subject, response-mode
preservation, NARRATIVE/EXAMPLE/existing-DEFINITION rules unchanged.

Lane re-measurement (multi_sentence_response, 21 cases):

  flag off: multi=0.1429, primed_multi=0.0000, conn=0.5385, grounded=0.8571
  flag on : multi=0.9048, primed_multi=1.0000, conn=0.8462, grounded=0.8571

Combined lift over the original (pre-wiring) baseline:
* multi_sentence_rate:        +70pp on the substantive predicate
* primed_multi_sentence_rate: +50pp (0.5 → 1.0 post-classifier)
* connective_present_rate:    +74pp (0.10 → 0.85)
* grounded_rate:              +39pp (0.47 → 0.86)

Cognition eval byte-identical: public 100/100/91.7/100, holdout
100/100/83.3/100 — these prompts aren't in cognition cases, and the
new rules don't perturb any rule that fires for cognition prompts.

Conversational thread coherence unchanged.

docs/evals/discourse_runtime_baseline_2026-05-19.md updated with the
full delta table; the planner is now load-bearing across the warm
and cold pack/teaching paths and the lane measures real capability
rather than punctuation artifacts.
2026-05-19 12:07:08 -07:00
Shay
8d1aeec42f fix(evals): refine multi-sentence response predicate 2026-05-19 11:40:47 -07:00
Shay
d5a6e81b33 feat(adr-0067): cross-pack teaching chains — Plan Phase 4 closed
ADR-0064 bound each teaching corpus 1:1 to a single ratified pack;
chains whose subject + object resolved to different packs were
dropped at load time. Phases 1–3 ratified the per-pack DAGs needed
to lift that constraint safely.

ADR-0067 introduces a deliberately narrow cross-pack chain shape.
Each entry carries explicit subject_pack_id and object_pack_id
fields, and the loader verifies per-chain residency. Same-pack
entries are rejected as corpus-misfilings (anti-leakage). The
cross-pack composer is the fall-through after the in-pack composer,
so the cognition lane stays byte-identical.

Files:
- chat/cross_pack_grounding.py — CrossPackChain + loader +
  single-chain composer + multi-chain enumerators
- teaching/cross_pack_chains/cross_pack_chains_v1.jsonl — 5 seed
  chains (family×identity, parent×understanding, family×memory,
  identity×family, understanding×parent)
- chat/runtime.py — fall-through wiring in CAUSE/VERIFICATION
- chat/narrative_surface.py, chat/example_surface.py — merge
  cross-pack chains, per-chain pack-residency helpers
- tests/test_cross_pack_chains.py — 31 tests covering loader,
  surface, multi-chain access, runtime integration, in-pack
  precedence
- tests/test_narrative_example_intents.py — corpus-tag assertions
  widened to allow cross-pack aggregation

Verification:
- 31 new tests pass
- Curated lanes: smoke 67 / cognition 121 / teaching 17 / packs 6 /
  runtime 19 — all green
- Cognition eval byte-identical (public 100/100/91.7/100, holdout
  100/100/83.3/100)
- Full lane: 2098 passed, 2 skipped, 0 failed in 2:30
2026-05-18 17:22:43 -07:00
Shay
ce8226e9a2 feat(adr-0066): NARRATIVE + EXAMPLE intents with multi-clause composers (Phase 3.3 + 3.4)
Two new intent shapes + composers turn the runtime's corpus
density into operator-visible articulation.  Both consult the
cross-corpus aggregator from ADR-0064; no new ratification needed.

P3.3 — chat/narrative_surface.py + IntentTag.NARRATIVE.

  Classifier patterns (registered BEFORE generic DEFINITION):
    ^tell\s+me\s+about\s+
    ^describe\s+
    ^what\s+(?:can|do)\s+you\s+(?:say|know)\s+about\s+

  narrative_grounded_surface(subject, max_clauses=4) walks every
  reviewed chain rooted on subject across all registered teaching
  corpora.  Dedupes by (connective, object) — cause + verification
  carrying the same predicate emit one clause, not two.  Sorts by
  (intent, connective, object) for replay stability.

  Surface format:
    "{X} — narrative-grounded ({corpus_ids}): {dX1}; {dX2}.
     {X} {conn1} {O1} ({dO1}); {X} {conn2} {O2} ({dO2}).
     No session evidence yet."

  Cross-corpus subjects (e.g. mother in relations_v2) emit
  narrative-grounded (relations_chains_v2) tag; cognition subjects
  emit cognition_chains_v1 tag.  Multi-corpus subjects (when
  applicable) emit composite "corpus_a + corpus_b" tag.

P3.4 — chat/example_surface.py + IntentTag.EXAMPLE.

  Classifier patterns:
    ^(?:give|show)\s+(?:me\s+)?an?\s+(?:example|instance)\s+of\s+
    ^example\s+of\s+

  example_grounded_surface(object_lemma, max_examples=3) walks chains
  where the lemma is the OBJECT — inverts the typical subject-keyed
  access pattern.  Dedupes by subject; sorts by (intent, subject,
  connective).

  Surface format:
    "{X} — example-grounded ({corpus_ids}): {dX1}.
     Example: {subj1} {conn1} {X}; {subj2} {conn2} {X}.
     No session evidence yet."

Cross-cutting:
  - Both intents added to _OOV_INTENT_TAGS — fall through to OOV
    invitation when subject is unknown (Phase 2 gradient discipline).
  - Both tagged grounding_source="teaching" (same provenance tier
    as the existing teaching_grounded_surface).
  - No prose generation, no new mutation surface.

Live verification:
  > Tell me about truth.
    [teaching] truth — narrative-grounded (cognition_chains_v1):
    cognition.truth; logos.core. truth grounds knowledge
    (cognition.knowledge); truth requires evidence (cognition.evidence).

  > Give me an example of knowledge.
    [teaching] knowledge — example-grounded (cognition_chains_v1):
    cognition.knowledge. Example: truth grounds knowledge;
    understanding requires knowledge; evidence grounds knowledge.

  > Tell me about mother.
    [teaching] mother — narrative-grounded (relations_chains_v2):
    kinship.parent.female. mother precedes daughter (kinship.child.female).

  > Describe photosynthesis.
    [oov] I haven't learned 'photosynthesis' yet (intent: narrative). ...

ADR-0066 (this commit completes the ADR).  30 new tests passed.
Full lane: 2067 passed, 2 skipped, 0 failed in 2:32.
2026-05-18 17:01:55 -07:00
Shay
ea298bdc28 feat(teaching): OOV signal flywheel — sink, aggregator, auto-promotion (Phase 2.3)
Mirrors the chain-gap pipeline (Phase 1.1+1.2) for vocabulary gaps:
the OOV invitation surface (P2.1) now emits structured signals that
operators can aggregate, rank, and auto-promote into reviewed
PackMutationProposal candidates — closing the OOV loop the same way
Phase 1 closed the chain loop.

Three new modules + two new CLI surfaces:

teaching/oov_sink.py.
  OOVCandidate dataclass mirroring teaching.discovery.DiscoveryCandidate.
  OOVBufferSink (in-memory) + OOVMonthlyFileSink (append-only JSONL
  under <root>/<YYYY>/<YYYY-MM>.jsonl — same layout as discovery sink
  so the aggregator reuses the file-walk machinery).
  hash_oov_candidate_id(token, intent, trace_hash) — deterministic
  32-char hex id matching DiscoveryCandidate's replay invariant.
  format_oov_candidate_jsonl — sorted-keys compact JSONL line.

teaching/oov_gaps.py.
  aggregate_oov_gaps(root, since, sample_limit) groups emitted
  candidates by token, tracks intent-shape union (a token asked under
  multiple intents is a stronger curriculum signal), splits
  boundary_clean from boundary_tainted counts, supports --since
  YYYY-MM filtering via the sink's file naming convention.
  Pure reader; never mutates the sink.  Deterministic ordering:
  (count desc, token asc).

teaching/oov_promotion.py.
  promote_oov_gaps(gaps, threshold, include_tainted, suggested_packs)
  lifts threshold-crossing tokens to OOVPromotion records.
  - boundary_clean_count gates promotion by default (tainted-only
    tokens may indicate the prompt hit a safety axis rather than a
    vocab gap).
  - --include-tainted flag for operator override.
  - threshold < 1 raises.
  - queue_id deterministic: ``oov:<token>@<threshold>`` — diffable
    across runs.
  - suggested_packs lists mounted packs but does NOT recommend one
    — domain inference is out of scope (would require a stochastic
    classifier).  Operator picks the destination.

Runtime wiring:
  ChatRuntime.attach_oov_sink(sink) mirrors attach_discovery_sink.
  Runtime emits one OOVCandidate JSONL line per turn whose
  grounding_source == "oov", no-op when no sink is attached.
  Intent classifier is now invoked when EITHER sink is attached
  (was: only discovery sink) — both downstream paths need it.

CLI:
  core teaching oov-gaps [--top N] [--since YYYY-MM] [--root PATH]
                          [--sample-limit N] [--json]
  core teaching oov-queue [--threshold N] [--include-tainted]
                          [--root PATH] [--since YYYY-MM] [--json]

ADR-0065 documents the full design (five-tier honesty gradient,
P2.1-P2.4 architecture).  README.md updated with the ADR-0065
index entry.

Verification:
  tests/test_oov_pipeline.py                      24 passed
  Operator workflow round-trip verified live:
    > rt.attach_oov_sink(sink); rt.chat("What is photosynthesis?")
    → sink receives:
      {"boundary_clean":true,"candidate_id":"f51bf8...",
       "intent":"definition","token":"photosynthesis","trigger":"unresolved_subject",
       "source_turn_trace":"","review_state":"unreviewed"}
    > core teaching oov-gaps --root /tmp/oov_demo
    → ranked table by count, intent-set per token
    > core teaching oov-queue --root /tmp/oov_demo --threshold 2
    → promoted tokens + suggested mounted packs

Full lane: 2005 passed, 2 skipped, 0 failed in 2:34 (xdist).
2026-05-18 16:42:26 -07:00
Shay
b5ba9b6d6f feat(adr-0064): cross-pack teaching chains + relations_chains_v1 seed (Phase 1.3+1.4)
ADR-0064 is the corpus-layer sibling of ADR-0063.  The teaching-grounded
surface composer was hardcoded to cognition_chains_v1, so kinship CAUSE/
VERIFICATION prompts fell through to the universal disclosure even though
en_core_relations_v1 was mounted on the live runtime (ADR-0063).

Architectural change in chat/teaching_grounding.py:

  - New TeachingCorpusSpec dataclass (corpus_id, path, pack_id).
  - TEACHING_CORPORA tuple registers every active corpus.  Each
    corpus is 1:1-bound to one lexicon pack — cross-domain triples
    deferred per docs/teaching_order.md §5.
  - _load_corpus(spec) loads one corpus with pack-residency scoped
    to its declared pack.
  - _all_chains_index() aggregates across all registered corpora
    (first-match-wins; cognition first preserves byte-identity).
  - _pack_for_corpus(corpus_id) → bound pack lexicon.
  - clear_teaching_caches() atomic cache invalidation.
  - TeachingChain gains corpus_id field → surface tag follows resolving corpus.

Wiring updates:

  - teaching_grounded_surface + teaching_grounded_surface_composed
    consult _all_chains_index; surface tag follows chain.corpus_id.
  - teaching/discovery.py gate uses chat.pack_resolver.is_resolvable
    (any mounted pack) + _all_chains_index (any registered corpus).
  - teaching/replay.py _swap_corpus_path rewrites the registry path
    + clears all teaching caches during the gate's transient phase.
    Active corpus bytes unchanged (replay invariant preserved).
  - evals/learning_loop/run_demo.py scene-5 swap mirrors the new
    pattern so the demo still grounds against transient corpora.

Back-compat preserved: _corpus_index, _CORPUS_PATH, TEACHING_CORPUS_ID
remain cognition-corpus-specific for audit/replay consumers.

Phase 1.4 — relations_chains_v1 seeded with 7 reviewed kinship chains:
  cause_parent_precedes_child
  cause_child_follows_parent
  cause_ancestor_precedes_descendant
  cause_descendant_follows_ancestor
  cause_family_grounds_parent
  verification_child_requires_parent
  verification_descendant_requires_ancestor

5 of 8 relations lemmas covered.  All connectives already humanised.
Strict pack-internal to en_core_relations_v1 (no cross-domain in v1).
Seed pattern matches cognition_chains_v1's original pre-ADR-0055 seed.

Live verification:
  > Why does parent exist?
  parent — teaching-grounded (relations_chains_v1):
  kinship.ascendant.direct; kinship.parent.
  parent precedes child (kinship.descendant.direct).
  grounding_source = teaching

Cognition eval byte-identical to pre-ADR baseline:
  public:  intent 100% / surface 100% / term 91.7% / closure 100%
  holdout: intent 100% / surface 100% / term 83.3% / closure 100%

Lanes green: smoke 67 / cognition 121 / teaching 17 / packs 6 /
runtime 19 / algebra 132 / full 1933 passed.
2026-05-18 16:04:20 -07:00
Shay
9f83b27a7c feat(adr-0063): cross-pack surface resolver — kinship lemmas ground on live path
ADR-0063 closes the ADR-0048/0050/0053/0061 hardcoded-cognition-pack
asymmetry. New chat/pack_resolver.py provides resolve_lemma(lemma,
pack_ids) → (resolving_pack_id, semantic_domains) across an ordered
tuple of mounted lexicon packs (first-match-wins, lru_cache per-pack).

Surface composers in chat/pack_grounding.py now consult the resolver
instead of a hardcoded en_core_cognition_v1. en_core_relations_v1
joins RuntimeConfig.input_packs defaults; kinship lemmas now ground
on the live path:

  > What is a parent?
  parent — pack-grounded (en_core_relations_v1):
  kinship.ascendant.direct; kinship.parent; biology.progenitor.
  No session evidence yet.

Cross-pack comparison (knowledge × parent) renders composite tag
(en_core_cognition_v1 × en_core_relations_v1). Cognition lane
remains byte-identical: cognition is resolved first and the surface
format for cognition lemmas is unchanged.

Cognition eval (byte-identical to pre-ADR baseline):
  public  → intent 100% / surface 100% / term 91.7% / closure 100%
  holdout → intent 100% / surface 100% / term 83.3% / closure 100%

Curated lanes green: smoke 67 / cognition 121 / teaching 17 /
packs 6 / runtime 19 / algebra 132.

New tests: test_pack_resolver.py (28) + test_cross_pack_grounding.py
(17). test_en_core_relations_v1_pack.py: default-input-packs guard
inverted. test_pack_grounding.py: two stale ADR-0048 tests rewritten
(premises invalidated by ADR-0052/0061; now use fully-out-of-pack
prompts).

chat/teaching_grounding.py UNCHANGED — cognition_chains_v1 corpus
stays cognition-only. Cross-pack teaching corpora are the natural
ADR-0064.
2026-05-18 15:00:58 -07:00
Shay
f0c57eb32e feat(packs): en_core_relations_v1 — kinship starter pack (8 lemmas)
Per teaching_order.md §5 — pick one commercial domain and run the
full 1→4 progression inside it before opening a second.  Kinship is
the doctrinally classic starter: tight DAG, well-bounded primitives,
and orthogonal to the cognition pack.

Lemmas (8): parent, child, sibling, family, ancestor, descendant,
spouse, offspring.  Each carries ≥2 semantic_domains under a
deterministic taxonomy (kinship.*, lineage.*, biology.*, social.*).

Deliberate exclusions:
  - `person` — lives in en_core_cognition_v1; orthogonality test
    pins that boundary.
  - Specializations (mother/father/son/daughter/grandparent/...) —
    derived from v1 primitives; land in v2 after v1 produces
    reviewed chains.
  - Quantifiers (one/two/many) — separate domain
    (en_core_quantification_v1); cross-domain triples come last.
  - Verbs of relation (begets/marries/...) — separate composer
    work; no relations_chains_v1.jsonl yet.

Engagement is opt-in:
  - Pack is NOT in RuntimeConfig.input_packs defaults.
  - Programmatic mount via RuntimeConfig(input_packs=(..., "en_core_relations_v1")).
  - CLI: core chat --pack en_core_relations_v1 (existing surface).
  - Default-not-mounted preserves the cognition lane unchanged
    until cross-pack teaching-grounded composition exists.

- language_packs/data/en_core_relations_v1/lexicon.jsonl
  — 8 entries, JSONL format matching en_core_cognition_v1.
- language_packs/data/en_core_relations_v1/manifest.json
  — pack_id, language, role=operational_base, checksum
  (SHA-256 of lexicon bytes per CLAUDE.md pack-discipline),
  version 1.0.0, determinism_class D0, oov_policy tagged_fallback.
- tests/test_en_core_relations_v1_pack.py — 6 tests pin:
  checksum-match load, lemma roster, per-lemma primary domain,
  ≥2 domains/lemma (composer headroom), zero collision with
  cognition pack (kinship DAG stays orthogonal), pack-not-in-
  default-input-packs (opt-in engagement contract).
- docs/curriculum/relations_pack_v1.md — full pack log:
  rationale per included/excluded lemma, opt-in engagement path,
  4-step ADR roadmap (cross-pack composition → first kinship
  chains → pronoun v2 → cross-domain triples).

Mounted-manifold sanity check (en_core_cognition_v1 +
en_core_relations_v1): 93 lemmas combined, no collisions, both
packs' surfaces individually addressable.

Lanes (regression): smoke 67 / packs 6 / algebra 132 / relations-pack 6.
The non-negotiable field invariant (versor_condition < 1e-6) is
unaffected: this is pure pack data + a contract test.
2026-05-18 14:40:54 -07:00
Shay
c492014815 feat(adr-0062): composed teaching-grounded surface (chain-of-chains)
Pre-ADR-0062, the teaching-grounded composer emitted exactly one
reviewed chain per surface — "light reveals truth" — even when the
corpus already contained an immediate follow-up "truth grounds
knowledge".  With 21 active chains after curriculum saturation v2,
many grounded prompts had a corpus-ratified follow-up the composer
silently dropped.

ADR-0062 adds the composed composer + an opt-in config flag:

  flag OFF (default):
    light — teaching-grounded (cognition_chains_v1): cognition.illumination;
    logos.core. light reveals truth (cognition.truth). No session evidence yet.

  flag ON:
    light — teaching-grounded (cognition_chains_v1): cognition.illumination;
    logos.core. light reveals truth (cognition.truth), which grounds
    knowledge (cognition.knowledge). No session evidence yet.

Follow-up resolution:
  - prefer cause; fall back to verification (deterministic preference)
  - cycle guard: 1-step cycles (A→B, B→A) blocked
  - pack-residency guard: follow-up's object must be pack-resident
  - bounded depth: v1 follows exactly one hop
  - degrades to single-chain BYTE-IDENTICALLY when no follow-up
    survives the guards (drop-in replacement)

Trust-boundary invariants preserved:
  - Every visible non-template token is lemma / pack-domain /
    humanize_predicate connective / template constant.  Only added
    template constant: ", which "
  - Deterministic: same chains → same surface bytes
  - Default-False flag pattern mirrors ADR-0047/0058
  - `versor_condition < 1e-6` invariant untouched (surface composition only)

Cognition lane null-drop invariant CI-pinned:
  Composed mode emits a strictly LONGER surface (extra follow-up
  clause); every expected_term passing flag-OFF must still pass flag-ON.
  Asserted in test_cognition_lane_metrics_unchanged_with_composed_flag
  for both public and holdout splits.  If a future change drops tokens,
  the test fails as a deliberate regression.

  public  flag OFF: intent 100% / surface 100% / term 91.7% / versor 100%
  public  flag ON : intent 100% / surface 100% / term 91.7% / versor 100% (identical)
  holdout flag OFF: intent 100% / surface 100% / term 83.3% / versor 100%
  holdout flag ON : intent 100% / surface 100% / term 83.3% / versor 100% (identical)

Live-prompt lift visible on ~12 of 21 active chains; the rest hit
cycle or pack-residency guards.  Saturation v2's clusters were
authored partly with composition in mind (thought→meaning→
understanding, inference→evidence→knowledge, etc.).

- core/config.py — `RuntimeConfig.composed_surface: bool = False`
- chat/teaching_grounding.py — `teaching_grounded_surface_composed`
  sibling to `teaching_grounded_surface`
- chat/runtime.py — dispatch branch in `_maybe_pack_grounded_surface`
  selects composed vs single-chain based on config flag
- tests/test_composed_surface.py — 11 tests pin: function-level
  (None on no chain / degrades when no follow-up / two-clause when
  follow-up exists / includes intermediate + final domains /
  deterministic / cycle guard / trust label preserved); runtime
  integration (default single-chain / flag-on composed / frozen
  config); cognition-lane null-drop invariant.

Lanes (regression): smoke 67 / cognition 121 / teaching 17 /
composed-surface 11 — all green.
2026-05-18 14:34:45 -07:00
Shay
a0edbb4bdb curriculum(cognition-saturation-v2): seven reviewed chains; pack coverage 14→21
Second curriculum unit through the production operator surfaces.
Pure saturation — no cognition-lane lift expected (the eval splits
test fixed 32 cases that don't overlap with this unit's subjects),
but the live-prompt grounding surface expands materially: seven
prompts that previously fell through to disclosure now route to
deterministic teaching-grounded surfaces.

Three coherent clusters:

  A. Cognition-source
     cause_thought_reveals_meaning
     cause_question_reveals_understanding
     cause_recall_reveals_memory

  B. Conceptual structure (bidirectional)
     cause_definition_grounds_concept
     verification_concept_requires_definition

  C. Semantic content
     cause_meaning_grounds_understanding
     cause_analogy_reveals_relation

All pack-consistent (subject + object in en_core_cognition_v1),
canonical predicates (reveals / grounds / requires), each opens a
previously-empty (subject, intent) cell.

Replay-equivalence gate reported replay_equivalent=True for all
seven proposals (public cognition lane byte-identical pre/post
every accept).

Cognition lane:
  public  : intent 100% / surface 100% / term 91.7% / versor 100%   (unchanged)
  holdout : intent 100% / surface 100% / term 83.3% / versor 100%   (unchanged)

Saturation lift is visible at the live-prompt level, not at the
eval level:

  Why does thought exist?              → [teaching] thought reveals meaning (...)
  Why does a question exist?           → [teaching] question reveals understanding (...)
  Why does definition exist?           → [teaching] definition grounds concept (...)
  Why does meaning exist?              → [teaching] meaning grounds understanding (...)
  Why does an analogy exist?           → [teaching] analogy reveals relation (...)
  Does a concept require definition?   → [teaching] concept requires definition (...)
  Why does recall exist?               → [teaching] recall reveals memory (...)

Why saturation matters: the cognition pack has 78 lemmas; we've
now covered ~21 (subject, intent) cells of the hundreds available.
Without saturation, prompts outside the 32 fixed eval cases are
coin-flips between vault recall and disclosure.  Saturation moves
marginal prompts to deterministic teaching-grounded surfaces — the
foundation the composed-surface ADR (next) will compose over.

- teaching/cognition_chains/cognition_chains_v1.jsonl — 15 → 22 lines
  (7 appends).  Active set: 14 → 21 chains.
- teaching/proposals/proposals.jsonl — 7 new (created → replay →
  transition → accepted_corpus_append) event sequences appended.
- docs/curriculum/cognition_saturation_v2.md — full curriculum log:
  cluster rationale, live-prompt lift, operator-wall-time profile,
  saturation-state-of-the-pack.

Lanes (regression check):
  core test --suite smoke           67 passed
  core test --suite cognition      121 passed
  core test --suite teaching        17 passed

The non-negotiable field invariant (versor_condition < 1e-6) is
unaffected: this is corpus growth only; no code path changed.
2026-05-18 14:29:30 -07:00
Shay
bf7f7895fe feat(adr-0061): PROCEDURE intent routes to pack-grounded surface
Pre-ADR-0061 every "How do I X?" question fell through to the
universal disclosure even when X was a pack-resident lemma.  The
teaching corpus carries CAUSE/VERIFICATION chains only — procedural
knowledge is fundamentally different in kind from propositional
claims and deserves its own ratification path (deliberately out of
scope; a future parallel `procedure_chains_v1.jsonl` schema is
discussed in the ADR's non-goals).

ADR-0061 adds the honest cold-start fallback: ground the topic in
pack semantic_domains and note explicitly that ratified step-by-step
guidance does not exist yet.

Surface format:
  "procedure-grounded ({pack_id}): {lemma} ({d1}; {d2}).
   Step-by-step guidance for {lemma} is not yet ratified
   in this session."

Selector — **last** pack-resident lemma in the verb-phrase subject:
  "define a concept" → concept    (object beats verb)
  "verify a claim"   → verify     (verb wins when object is OOV)
  "correct an error" → correct
  "learn this"       → learn
  "do stuff"         → None       (falls through to universal disclosure)

Stopwords: only `be` and `have` (dialogue fillers).  Procedure verbs
are deliberately NOT stopworded so the verb-as-fallback rule fires
when the object is OOV — keeps surface coverage.

Trust-boundary invariants:
  - Every visible non-template token is lemma / pack-domain / template.
  - Deterministic: same subject_text → same bytes.
  - Returns None for fully-unknown utterances → universal disclosure
    fires.  Never fabricates surface from nothing (ADR-0053 contract).
  - "not yet ratified" trust-label preserved.

Cognition lane lift:
  public  : intent 100% / surface 100% / term 91.7% / versor 100%      (unchanged)
  holdout : intent 100% / surface 94.7%→100.0% / term 79.2%→83.3% / versor 100%

Two cases fixed:
  - procedure_define_010 ("How do I define a concept?") — surface +
    term `concept` now captured.
  - procedure_verify_034 ("How do I verify a claim?") — surface only
    (case has no expected_terms; the verb fallback grounds it).

Combined effect: holdout `surface_groundedness` closes to 100%; 4 of
5 architectural holdout misses now resolved (this ADR + ADR-0060 +
the supersede from epistemology v1).  Remaining 2 are UNKNOWN-intent
cases (unknown_spirit_041, unknown_word_018) — out of scope; deserve
their own ADR with distinct selector semantics.

- chat/pack_grounding.py — `_extract_procedure_topic_lemma` helper +
  `pack_grounded_procedure_surface` composer.
- chat/runtime.py — import + dispatch branch for `IntentTag.PROCEDURE`.
- tests/test_procedure_surface.py — 15 tests pin: extraction
  (last-wins / verb-by-elimination / be+have skipped / None on empty /
  strips punctuation / case-insensitive); surface (contains lemma /
  contains domains / pack_id / "not yet ratified" label / None for
  no-pack-lemma / deterministic); end-to-end through ChatRuntime.

Lanes (regression): smoke 67 / cognition 121 / teaching 17 /
procedure 15 — all green.

The non-negotiable field invariant (versor_condition < 1e-6) is
unaffected: this ADR changes surface composition only.
2026-05-18 14:22:19 -07:00
Shay
c9e858c266 feat(adr-0060): correction acknowledgement carries corrected-topic lemma
ADR-0053's cold-start CORRECTION surface was topic-blind: a user who
said "Actually, truth requires evidence" got a response referencing
`correction` but never `truth`.  The holdout case correction_truth_040
expected `term=['truth']` and missed — one of the architectural gaps
surfaced by the epistemology v1 curriculum unit.

ADR-0060 closes that gap by weaving the first pack-resident topical
lemma from the utterance into a fixed-template extension:

  correction received — pack-grounded ({pack_id}):
  {correction_domains}. Noted topic: {lemma} ({lemma_domains}).
  No prior turn in this session to correct yet.

Selection rule (deterministic, left-to-right token order):
  - skip stopwords: `correction`, `correct`, `be`, `have`
  - pick first pack-resident lemma
  - if none found → ADR-0053 topic-less template byte-identically

Trust-boundary invariants preserved:
  - Every visible non-template token is still lemma / pack-domain / template
  - Deterministic: same text → same bytes
  - Backward compatible: existing 15 ADR-0053 tests pass byte-identically
  - "No prior turn in this session to correct yet." trust label kept

Cognition lane lift:
  public  : intent 100% / surface 100% / term 91.7% / versor 100%   (unchanged)
  holdout : intent 100% / surface 94.7% / term 75.0%→79.2% / versor 100%

The +4.2pp matches the single-case fix exactly (correction_truth_040).
Remaining 3 holdout misses (procedure_define_010, unknown_spirit_041,
unknown_word_018) are out of scope for this ADR.

- chat/pack_grounding.py — `_extract_correction_topic_lemma` helper +
  optional `text` parameter on `pack_grounded_correction_surface`.
- chat/runtime.py — single-line call-site change to pass `text` through.
- tests/test_correction_topic_lemma.py — 14 new tests pin:
  extraction (first lemma / skips correction / skips fillers / None on
  empty / strips punctuation / case-insensitive); surface (contains
  corrected lemma / contains topic domains / degrades to ADR-0053
  byte-identically / preserves trust label / deterministic / correct
  pack_id); end-to-end (correction_truth_040 emits 'truth' / no-pack-
  lemma still grounds).

Why text-level extraction, not intent.subject:
  `intent.subject` after ADR-0049 head-noun extraction returns
  ", truth requires evidence" for the test prompt — the CORRECTION
  intent's subject-extractor preserves the post-marker tail.  Parsing
  the raw text at the surface layer is cleaner; isolates the fix;
  doesn't perturb upstream classification logic.

Lanes (regression): smoke 67 / cognition 121 / teaching 17 /
correction tests 29 (15 ADR-0053 backward-compat + 14 ADR-0060 new) —
all green.

The non-negotiable field invariant (versor_condition < 1e-6) is
unaffected: this ADR changes surface composition only.
2026-05-18 14:14:27 -07:00
Shay
2acf71f024 curriculum(epistemology-v1): five reviewed chains; holdout term_capture +4.2pp
First end-to-end curriculum unit through the production
propose / review --accept / supersede operator surfaces against the
active teaching corpus.  Replay-equivalence gate passed for every
proposal; public split byte-identical; holdout term_capture lifted
exactly as predicted.

- Supersede `verification_wisdom_grounds_judgment` →
  `verification_wisdom_requires_knowledge`.  Fixes the only corpus-
  fixable holdout miss: `verification_wisdom_036`
  ("Is wisdom the same as knowledge?") now grounds with both
  expected terms.  Provenance carries
  `:supersede(verification_wisdom_grounds_judgment)`.
- Propose + accept four new chains closing epistemology subgraph
  cells:
    cause_understanding_requires_knowledge
    cause_judgment_requires_wisdom
    verification_evidence_grounds_knowledge
    cause_inference_requires_evidence

Each chain is pack-consistent, uses canonical predicates, and opens
a previously-empty (subject, intent) cell.  Replay gate confirmed
no metric regression on the public split before each accept.

Lift (cognition eval):
  public  : intent 100% / surface 100% / term 91.7% / versor 100%   (unchanged)
  holdout : intent 100% / surface 94.7% / term 70.8%→75.0% / versor 100%

The remaining four holdout misses (correction_truth_040,
procedure_define_010, unknown_spirit_041, unknown_word_018) are
architectural — surface-composition gaps in the correction-
acknowledgment template, procedure-intent routing, and unknown-
intent surface — and out of scope for corpus surgery.

- teaching/cognition_chains/cognition_chains_v1.jsonl — 10 → 15 lines
  (4 appends + 1 supersession marker; 1 retired chain still on disk
  per the audit doctrine of append-only at the file level).
- teaching/proposals/proposals.jsonl — new append-only proposal log
  with `created` / `replay` / `transition` / `accepted_corpus_append`
  events for every accepted proposal.
- docs/curriculum/epistemology_v1.md — full curriculum log:
  rationale per chain, prediction-vs-result on the holdout lift,
  reproducibility commands, architectural-gap analysis.

Lanes (regression check):
  core test --suite smoke           67 passed
  core test --suite cognition      121 passed
  core test --suite teaching        17 passed
  tests/test_eval_holdout_split    10 passed

The first curriculum unit that *measurably moves a cognition-lane
metric* through the operator surfaces, with full provenance from
operator note back to corpus append.
2026-05-18 14:02:37 -07:00
Shay
29449f3775 feat(adr-0059): correction-pass telemetry emission — backward perturbation auditable
`ChatRuntime.correct()` propagates a backward perturbation through the
session graph (per session/correction.py): each past turn whose output
versor has non-trivial CGA-alignment with the correction versor is
blended toward it (decayed by graph distance).  The forward regen turn
that followed already emitted a TurnEvent — but the backward
perturbation itself was invisible to the telemetry sink.

ADR-0059 closes that gap with a discriminated event line.

- chat/telemetry.py — adds `serialize_correction_event` +
  `format_correction_event_jsonl` emitting one JSONL line discriminated
  by `"type": "correction"`.  Payload: target_turn, records_count,
  turns_skipped, turn_idxs_affected, max_delta_norm, mean_delta_norm,
  SHA-256 correction_versor_digest, pack ids.  No raw versor coordinates.
- chat/runtime.py — `_emit_correction_event` (mirrors
  `_emit_turn_event`); called from `correct()` after the graph state
  is updated but before the forward regen turn.  No-op without sink.
- tests/test_correction_telemetry.py — 7 tests pin: no-op without
  sink, emission with sink, payload shape (required keys + types +
  ranges), SHA-256 digest shape, trust boundary (no versor
  coordinates leaked), determinism (byte-identical lines across
  runs), correction event and turn event coexist in the sink.

Trust boundary (per CLAUDE.md):
  - Metadata-only: only L2 deltas + SHA-256 digest.
  - No implicit wall-clock.
  - Deterministic: same CorrectionResult → byte-identical line.
  - Sink contract unchanged: `emit(line: str)`.
  - `versor_condition < 1e-6` invariant: untouched (telemetry-only).

Verification: smoke 67 / runtime 19 / correction telemetry 7 — green.
2026-05-18 13:47:48 -07:00
Shay
fd80da6ac0 docs(adr-0058): forward_graph_constraint engaged-but-inert; null-lift pinned
ADR-0058 closes the ADR-0047 follow-up question ("should the
forward_graph_constraint flag become default-on or pack-opt-in?")
with the explicit answer: neither, yet.

The ADR-0047 A/B characterisation found that the flag is observably
inert on every public-cognition-lane metric — narrowing which tokens
the walk may visit did not change which surface gets emitted.  That
finding scoped ADR-0048..0053, which closed the cognition lane to
100.0% surface_groundedness / 91.7% term_capture_rate via realizer /
surface-assembly work downstream of propagation.

This ADR makes three load-bearing decisions:

  1. `forward_graph_constraint` remains opt-in with default `False`.
     No identity pack (including precision_first_v1) opts in.
  2. No `runtime_preferences` block is added to identity packs; no
     path from pack JSON to RuntimeConfig is opened.  Deferring the
     pack-to-runtime composition layer until at least one such
     preference has demonstrated lift avoids letting the wiring lead
     the lift and locking in an abstraction at the wrong level.
  3. The ADR-0047 null-lift finding is promoted from a historical
     observation to a CI-enforced invariant.  A new regression test
     runs the public cognition split twice (flag OFF vs ON) and
     asserts every watched metric is pair-wise identical.  If
     downstream realizer work later moves a metric on the flag flip,
     the test fails as a deliberate transition rather than silent drift.

- docs/decisions/ADR-0058-forward-graph-constraint-status.md — ADR doc.
- docs/decisions/README.md — index entry.
- tests/test_forward_graph_constraint_null_lift.py — 2 tests:
  null-lift invariant across the cognition lane, default-False contract.

Verification:
  smoke 67 passed; flag tests 7 passed (5 wiring + 2 null-lift).
  No runtime behaviour change; versor_condition < 1e-6 invariant unaffected.
2026-05-18 13:36:37 -07:00
Shay
763ed16d1c docs(adr-0055-0057): writeups + asciinema captures for the demo trilogy
Three shareable demo / benchmark writeups modeled on the existing
`docs/evals/phase6_comparative_demo.md` treatment, each accompanied
by an asciinema-rendered GIF for at-a-glance viewing on the repo page.

- docs/evals/anti_regression_demo.md — three-gate defense; per-gate
  table; honesty paragraph about the synthetic regression in S2 (real
  ReplayEvidence shape via documented run_replay= kwarg); sample run
  output; falsifiable claims index.
- docs/evals/learning_loop_demo.md — headline before/after; CORE-vs-
  pretraining comparison table; trust-boundary code snippet showing
  the _CORPUS_PATH swap; per-scene table; full sample run; subject-
  selection rationale (pack-resident ∧ no active chain ∧ deterministic
  intent classification).
- docs/evals/teaching_loop_bench.md — what's byte-identical and why
  it matters per artifact; 100-run reference numbers (unique=1 across
  all five artifacts; mean=1.849s p50=1.838s p95=1.851s); pairing
  paragraph with ADR-0045 (read vs write determinism).

GIF captures (rendered with asciinema 3.2.0 + agg 1.8.1, github-dark
theme, JetBrains Mono):
- docs/evals/assets/anti_regression.gif   (120K, 944x843)
- docs/evals/assets/learning_loop.gif     (332K, 944x1039)
- docs/evals/assets/teaching_loop_bench.gif (64K, 860x1000)

Raw .cast files preserved alongside the GIFs for re-rendering at
different themes / speeds / sizes without re-recording.

README.md — added writeup-link column to the Inter-Session Memory
three-demo table.
2026-05-18 11:18:56 -07:00
Shay
e03ab4b609 feat(adr-0057): Phase C2 — TeachingChainProposal + replay gate + review CLI
The only path by which CORE extends its own active teaching corpus.
Closes ADR-0055 Phase C alongside ADR-0056's cognitive surface.

Three load-bearing calls (recorded in ADR-0057):
  1. Replay-equivalence is a precondition, not a permission;
     operator --accept remains required.
  2. Eligibility = polarity in {affirms, falsifies} AND at least
     one source='corpus' evidence pointer AND boundary_clean AND
     claim_domain != evaluative (unless --allow-evaluative) AND
     proposed_chain complete.
  3. Append-only proposal log; corpus history append-only too.

Changes
- teaching/proposals.py — TeachingChainProposal, ReplayEvidence,
  ProposalLog (event-sourced replay → current_state), eligibility
  predicate, propose_from_candidate, accept/reject/withdraw,
  append_chain_to_corpus (the sole corpus-write surface).  Uses
  TYPE_CHECKING guards to break the circular import with
  chat.pack_grounding.
- teaching/replay.py — run_replay_equivalence; swaps _corpus_index
  path to a tmp file, runs cognition lane on the active corpus
  AND a transient copy with the proposed chain appended, returns
  regressed-metrics list; trust-boundary assertion that the active
  corpus bytes are byte-identical pre/post.
- teaching/discovery.py — moved chat.pack_grounding /
  chat.teaching_grounding imports inside extract_discovery_candidates
  to break the cycle (was masked when chat.runtime was the entry
  point; surfaced by CLI entry).
- core/cli.py — three new subcommands:
    core teaching propose <candidate-jsonl-path> [--allow-evaluative]
    core teaching proposals [--state pending|accepted|rejected|withdrawn] [--json]
    core teaching review <proposal_id> --accept --review-date YYYY-MM-DD
    core teaching review <proposal_id> --reject [--note ...]
    core teaching review <proposal_id> --withdraw [--note ...]
- tests/test_teaching_proposals.py — 16 tests covering: every
  eligibility gate, proposal_id idempotency, append-only log,
  replay-equivalent stays pending, regression auto-rejects with
  named regressed metrics, --accept appends one line with typed
  Provenance, --accept refused on non-equivalent, state-machine
  blocks double-accept, real replay gate runs cognition lane
  twice and asserts byte-clean active corpus pre/post.

Invariants preserved
- versor_condition(F) < 1e-6 — C2 touches no algebra path.
- Active corpus bytes byte-identical regardless of replay outcome.
- No clock-time reads, no LLM, no async.
- Proposal-only — accept_proposal is the sole corpus-write path.

Lanes: smoke 67 / cognition 121 / runtime 19 / teaching 17 /
new proposals 16.  Cognition eval unchanged.

Open follow-ups (not in scope):
- supersession via operator review action
- cross-pack falsification arbitration (ADR-0056 Call 2 deferred)
- pack-data migration of frame-dependent connectives

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 10:23:14 -07:00
Shay
f78def7f3a docs(adr-0056): mark Accepted in decisions index
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 10:10:53 -07:00
Shay
4e03a7f872 docs(adr-0056): Accepted (Phase C1 implemented at 4eecf73)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 10:07:27 -07:00
Shay
f1121b5822 docs(adr-0056): Contemplation loop C1 — Proposed
Splits ADR-0055 Phase C into:
- C1 (this ADR): cognitive contemplation loop — question
  decomposition + polarity (affirms/falsifies/undetermined) +
  claim_domain typing (factual/relational/evaluative)
- C2 (future ADR): review-and-apply — TeachingChainProposal,
  replay-equivalence gate, corpus append-on-accept

Documents four load-bearing design calls with explicit reasoning
so future sessions can re-derive without re-arguing:

1. Stopping condition: record-the-gap-and-stop primary, bounded
   depth failsafe; failsafe firing emits recursion_overflow audit
   signal — never silent truncation.
2. Falsification evidence: reviewed-only, same pack family;
   session-tier contests but does not falsify. Cross-pack
   arbitration deferred.
3. Order: C1 before C2. Reversed instinct to land 'small thing
   first' — C2 alone is useless without enriched input; C1
   physically cannot mutate corpus until C2 wires the apply path.
4. Sync, not async. CORE hot path is deterministic; concurrency
   overhead exceeds probe cost on local-only probes. Async
   deferred to a future ADR if a blocking probe surface emerges.

Trust boundary: C1 never mutates the corpus. C1 reads pack,
corpus, vault, and most recent TurnEvent; writes only to the
existing Phase B discovery sink. Gap-recorded sub-questions
emit as new top-level candidates on the same sink — recursion
reified into the stream.

Maps directly onto user-stated framing recorded verbatim in the
ADR:
- 'contemplation always starts with a question' → candidate is
  the posing; contemplate() is the answering
- 'truths and/or falsities' → polarity on the chain itself
- 'remain humble' → claim_domain with escalating evidence
  thresholds, mandatory hedge for evaluative
2026-05-18 08:52:43 -07:00
Shay
07d35c0f54 feat(adr-0055): Phase B — DiscoveryCandidate emission from turn loop
Lands the first deterministic trigger of the discovery → reviewed-
memory loop. Candidates are structured evidence; emission is
opt-in via attach_discovery_sink and NEVER mutates the active
teaching corpus.

- teaching/discovery.py: DiscoveryCandidate dataclass + pure
  extract_discovery_candidates(turn_event, intent, subject) rule
  firing. Phase B fires only the would_have_grounded trigger:
    grounding_source == "none"
    AND intent ∈ {CAUSE, VERIFICATION}
    AND subject lemma in ratified cognition pack
    AND (subject, intent) NOT in active corpus
  candidate_id = SHA-256 of canonical JSON payload — replay-stable.
  Other DiscoveryTrigger literals (successful_comparison,
  hedge_acknowledged, oov_resolved_via_decomp) are reserved for
  later phases.

- teaching/discovery_sink.py: DiscoveryCandidateSink protocol,
  DiscoveryBufferSink (in-memory), DiscoveryMonthlyFileSink
  (append-only JSONL, <root>/<YYYY>/<YYYY-MM>.jsonl rollover,
  injectable clock).

- chat/runtime.py: opt-in attach_discovery_sink, post-turn
  emission inside _stub_response only when caller threads
  classified intent forward (gate-fire fall-through site).
  Intent classification at the call site reuses the same
  deterministic classifier already invoked by
  _maybe_pack_grounded_surface for the empty-vault English path.

Trust boundary: candidates write to a separate sink/file path
only; the active corpus on disk is never touched. Tests
explicitly assert corpus bytes are byte-identical before and
after a candidate-emitting turn.

Tests: tests/test_discovery_candidates.py — 24 tests covering
pure-predicate rule firing, every short-circuit path,
deterministic candidate_id, sink opt-in, runtime parity with no
sink, monthly rollover semantics, append-only behaviour, no
corpus mutation.

Lanes: smoke 67, cognition 121, runtime 19, teaching 17, packs 6
— all green. Cognition eval metrics unchanged on dev / public /
holdout splits. versor_condition < 1e-6 invariant untouched.
2026-05-18 08:26:04 -07:00
Shay
0f797e2940 docs(adr-0055): inter-session memory — reviewed discovery promotion (Proposed)
Phased design for closing the inter-session learning loop without a
parallel learning path:

- Phase A: make today's 4-tier story load-bearing (audit CLI,
  active-set view via superseded_by, typed provenance enum)
- Phase B: DiscoveryCandidate emission from the turn loop —
  deterministic rule-firing on the audit trail, never writes the
  corpus
- Phase C: TeachingChainProposal — sibling to PackMutationProposal,
  proposal-only, replay-equivalence gate on dev+public
- Phase D: epistemic-tier guard (only COHERENT evidence promotes)
- Phase E: curriculum integration via formation review

Non-goals named explicitly: no embeddings, no DB storage, no
automatic identity/safety/ethics mutation, no opaque LLM step, no
removal of human reviewer.

Status Proposed; later ADRs land each phase against the verification
contracts named here.
2026-05-18 08:09:40 -07:00