Commit graph

70 commits

Author SHA1 Message Date
Shay
1feec74b1c feat(anchor_lens): ADR-0073d — L1.4 telemetry, CLI flag, tour demo
L1.4 closes the anchor-lens inside-out arc (L1.1→L1.4 mirroring
R1→R5).  Substantive axis is now operator-observable,
operator-driven, and demo-falsifiable — exactly what R5 did for
the register subsystem.

Telemetry extension
  - TurnEvent + ChatResponse gain anchor_lens_id +
    anchor_lens_mode_label (both default "" → pre-L1.4
    byte-identical).
  - serialize_turn_event surfaces both fields in every JSONL line.
  - Mode-label extracted via _ANCHOR_LENS_ANNOTATION_RE from the
    PRE-decoration surface (so register decoration cannot interfere
    with anchor-lens telemetry).  Composer remains the sole source
    of truth for engagement; the runtime helper is read-only.

Operator surface
  - core chat --anchor-lens <id> CLI flag threads into
    RuntimeConfig.anchor_lens_id.
  - Invalid id → AnchorLensError caught at cmd_chat and surfaced
    as _die("invalid --anchor-lens pack id: ...", code=2) before
    the REPL launches.
  - Composes with --register (both flags wire through
    _runtime_config_from_args).

Narrative demo
  - evals/anchor_lens_tour/run_tour.py walks 2 prompts × 3
    ratified lenses ({default_unanchored_v1, grc_logos_v1,
    he_logos_v1}).  Asserts four claims:
      * lens_ids_recorded_per_turn
      * trace_hashes_distinct_across_lenses (OPPOSITE of
        register-tour's identical-hash claim)
      * surface_propositions_distinct_across_lenses
      * no_substrate_glyph_leak (block-scoped Greek/Hebrew/
        Syriac/Arabic; stylistic punct allowed)
  - Exit code 0 iff all four hold.
  - Bundled into `core demo` choices + EPILOG.

Tests (30 new)
  - tests/test_anchor_lens_telemetry.py (16) — TurnEvent shape,
    serializer keys, runtime emits per lens / per engagement
    state, ChatResponse mirrors event, mode-label extractor unit.
  - tests/test_anchor_lens_cli.py (9) — _runtime_config_from_args
    threading, invalid id fail-fast, parser flag wiring, parser
    composes with --register.
  - tests/test_anchor_lens_tour_demo.py (9) — four seam claims
    pinned individually + all_claims_supported + per-cell
    anchor_lens_id + unanchored cells empty mode + engaged cells
    carry mode label.

Lane evidence
  - 30 new L1.4 tests pass.
  - core demo anchor-lens-tour --json → all_claims_supported: True.
  - core demo register-tour --json    → all_claims_supported: True.
    Both tours pass simultaneously — orthogonality CI-pinned.
  - python -m core.cli eval cognition → public 100/100/91.7/100
    byte-identical (lens=None / default_unanchored_v1).
  - Full lane: 2736 passed / 4 skipped / 1 pre-existing failure
    (+30 over L1.3's 2706; the one failure remains
    test_all_preamble_explains_combined_run, unrelated).

Live demo (canonical proof)
  P1: 'What is knowledge?'
    default_unanchored_v1  trace=17c9aabe…  mode=(none)
    grc_logos_v1           trace=0198ad4c…  mode=systematic
    he_logos_v1            trace=17c9aabe…  mode=(none)
  P2: 'What is truth?'
    default_unanchored_v1  trace=2557f3e8…  mode=(none)
    grc_logos_v1           trace=2557f3e8…  mode=(none)
    he_logos_v1            trace=ec8d84aa…  mode=covenant-verity

  Engagement is substrate-scoped: grc never touches truth, he
  never touches knowledge.  Trace hashes diverge exactly where the
  lens engages.

Trust boundaries
  - --anchor-lens flag does not bypass ratification; loader still
    enforces companion mastery report self-seal + ratify-time
    substrate-atom existence check (ADR-0073b/c).
  - Mode-label extraction is read-only regex parse; can't forge
    annotations the composer didn't emit.
  - Telemetry stays redact-safe — both fields are identifiers /
    mode labels, not content.  include_content=False emits them
    unconditionally.
  - No new mutation surface; pack files unchanged.

Closes the anchor-lens inside-out arc
  L1.1  content prerequisite                  ✓ (ADR-0073a)
  L1.2  class + loader + unanchored sentinel  ✓ (ADR-0073b)
  L1.3  first lenses + composer wiring        ✓ (ADR-0073c)
  L1.4  telemetry + CLI + tour demo           ✓ (this commit)

  Mirrors the R1→R5 register cadence exactly.  Both axes are now
  operator-observable, CI-falsifiable, audit-traceable, and
  composable via the orthogonality claim pinned in both tours.
2026-05-19 20:21:41 -07:00
Shay
b35bec6465 feat(anchor_lens): ADR-0073c — L1.3 first lenses + composer wiring
L1.3 of the anchor-lens inside-out rollout — first substantive
surface lift on the substantive axis.  Two ratified non-trivial
lenses engage on cognition-pack lemmas via the alignment graph,
appending [lens(<id>):<mode>] annotations to the existing
pack-grounded surface.

Two ratified lenses

  grc_logos_v1 (Greek substrate)
    primary_substrate         : "grc"
    semantic_domain_preferences: ["logos.episteme.systematic_knowledge"]
    cognitive_mode_label       : "systematic"
    Engages on en "knowledge" via grc-core-cog-021 (ἐπιστήμη) →
    en-core-cog-007 alignment edge.

  he_logos_v1 (Hebrew substrate)
    primary_substrate         : "he"
    semantic_domain_preferences: ["logos.aletheia.verity"]
    cognitive_mode_label       : "covenant-verity"
    Engages on en "truth" via he-core-cog-002 (אמת) →
    en-core-cog-002 alignment edge.

  Both ratified under method anchor_lens_lifts_proposition.

Engagement rule (single)

  1. Resolve en_lemma → entry_id (cognition pack).
  2. For each substrate pack matching lens.primary_substrate, load
     alignment.jsonl; find edges where target_id == entry_id.
  3. For each such substrate lemma, if any atom in its
     semantic_domains ∈ lens.semantic_domain_preferences → engage.
  4. No match → None (no annotation; byte-identical surface).

The pivot is shared semantic_domain atoms surfaced via the
alignment graph — exactly the language-neutral commitment from
ADR-0073.  Engagement never touches non-English surface text;
entry_ids and atom strings only.

Surface lift

  no-lens : "Knowledge is X. pack-grounded (en_core_cognition_v1)."
  lens-on : "Knowledge is X. pack-grounded (en_core_cognition_v1) [lens(grc_logos_v1):systematic]."

  Annotation between existing provenance and trailing period.
  Both metadata fields are ASCII-bounded ≤64 chars at the loader
  level, so the annotation can never carry non-ASCII.

Scope deliberately narrow

  L1.3 wiring restricted to pack_grounded_surface /
  build_pack_surface_candidate (DEFINITION/RECALL only).  Other
  composers (COMPARISON / CORRECTION / PROCEDURE / NARRATIVE /
  EXAMPLE / CAUSE / VERIFICATION) accept the anchor_lens kwarg via
  forward-compat default UNANCHORED but do not yet consume it.
  L1.3b or later broadens to those intent shapes.

Ratify gate widening

  Non-null lenses must:
    - have primary_substrate ∈ {grc, he, en}
    - have a non-empty cognitive_mode_label
    - every preferred atom must exist in at least one lemma of the
      named substrate (trust boundary: operators cannot ship a lens
      pointing at atoms not on disk).
  Method: anchor_lens_lifts_proposition.  Null lenses still ratify
  under byte_identity_null_lift (L1.2 method).

Seam allow-list widening

  Truth-path modules (cognition / trace / pipeline / intent /
  propagation / vault / algebra) still refused.  Composer-side
  imports from chat/pack_grounding.py now permitted — the same way
  ADR-0069's R2 widened the register seam.

New invariants pinned (3)

  tests/test_anchor_lens_engagement_unit.py (14 tests) — resolver
  returns mode label only on intended substrate × en lemma pair;
  case-insensitive; engagement None under null lens; synthetic
  lens with unmatched atom returns None; annotation is pure ASCII.

  tests/test_anchor_lens_lifts_proposition.py (17 tests) — grc
  engages on knowledge only, he engages on truth only,
  cross-lens isolation, three-way distinctness, replay determinism
  per (lens × prompt), register-tour seam holds within each lens
  scope (orthogonality CI-pinned, parametrized over 4 lens
  choices).

  tests/test_anchor_lens_no_glyph_leak.py (5 tests) — hard
  block-scoped gate: Greek (U+0370..03FF, U+1F00..1FFF), Hebrew
  (U+0590..05FF), Syriac, Arabic.  Stylistic punctuation
  (em-dash etc.) explicitly allowed; em-dash predates L1.3 by a
  wide margin and is not a substrate-leak risk.  Tested per-lens
  across every cognition case + direct lens-metadata ASCII check.

Lane evidence

  74 anchor-lens tests pass (37 from L1.2 + 37 new).
  python -m core.cli eval cognition → public 100/100/91.7/100
  byte-identical (lens=None / default_unanchored_v1).
  core demo register-tour --json → all_claims_supported: True
  (R5 seam still holds; L1.3 doesn't perturb presentation axis).
  Full lane: 2706 passed / 4 skipped / 1 pre-existing failure
  (+37 over L1.2's 2669; the one failure remains
  test_all_preamble_explains_combined_run, unrelated).

Files

  packs/anchor_lens/grc_logos_v1.json                        NEW
  packs/anchor_lens/grc_logos_v1.mastery_report.json         NEW
  packs/anchor_lens/he_logos_v1.json                         NEW
  packs/anchor_lens/he_logos_v1.mastery_report.json          NEW

  scripts/ratify_anchor_lens_packs.py                        EDIT
    LENS_IDS adds grc_logos_v1 / he_logos_v1; gate widened.

  chat/pack_grounding.py                                     EDIT
    _resolve_anchor_lens_mode, _maybe_append_anchor_lens_annotation,
    _substrate_lexicon_by_entry_id, _en_lemma_to_entry_id.
    build_pack_surface_candidate + pack_grounded_surface gain
    anchor_lens kwarg (default UNANCHORED).

  chat/runtime.py                                            EDIT
    Thread self.anchor_lens into pack_grounded_surface() call.

  tests/test_anchor_lens_pack_seam.py                        EDIT
    Doc-comment updated for L1.3 allow-list.

  tests/test_anchor_lens_*                                   NEW (3 files)

  docs/decisions/ADR-0073c-anchor-lens-composer-wiring.md    NEW
2026-05-19 20:06:02 -07:00
Shay
9b1b63b253 feat(anchor_lens): ADR-0073b — L1.2 class + loader + unanchored sentinel
L1.2 of the anchor-lens inside-out rollout — pack class, loader,
ratified sentinel pack, and runtime threading.  Mirrors the
ADR-0068 register-class pattern exactly.  No composer consumes the
lens yet — that's L1.3.

AnchorLens frozen dataclass (packs/anchor_lens/loader.py)
  - lens_id / version / description / display_name
  - primary_substrate ∈ {grc, he, en, none}
  - semantic_domain_preferences: tuple[str, ...] (ordered, ≤64 atoms
    of ≤64 chars each, no duplicates)
  - cognitive_mode_label: str (≤64 chars)
  - mastery_report_sha256
  - is_unanchored() / is_null_lens() predicates
  - unanchored() classmethod + module-level UNANCHORED singleton

Loader contract (mirror of packs/register/loader.py)
  - safe_pack_id path-traversal rejection
  - Schema validation + envelope bounds checks
  - Companion mastery report self-seal + report_sha256 verification
  - CORE_ALLOW_UNRATIFIED_ANCHOR_LENS=1 dev bypass
  - require_ratified default True
  - No truth-path imports (pinned by seam test)

default_unanchored_v1 ratified pack
  - Null lens: primary_substrate="none", empty preferences,
    empty cognitive_mode_label
  - Self-sealed at b3235072fdbb2219...
  - Ratification method: byte_identity_null_lift
  - scripts/ratify_anchor_lens_packs.py L1.2 gate accepts only
    null lenses; L1.3 will widen.  Idempotent.

RuntimeConfig threading
  - new field: anchor_lens_id: str | None = None
  - new constant: DEFAULT_ANCHOR_LENS = "default_unanchored_v1"
  - ChatRuntime.__init__ loads the lens (None → AnchorLens.
    unanchored(); otherwise load_anchor_lens(id)) and stores as
    self.anchor_lens + self.anchor_lens_id.  Invalid ids fail-fast
    at init via AnchorLensError, not at first turn.
  - No composer reads the attribute yet.

Tests pinned (37 total)
  - tests/test_anchor_lens_pack_loader.py (24) — load happy path,
    sentinel structural identity, invalid id rejection (traversal,
    empty, slashes, missing), ratification bypass paths, companion
    SHA mismatch, bounds (substrate / preferences / atoms / label /
    duplicates / capacity), field-missing, lens_id mismatch with
    filename, unsupported schema_version.
  - tests/test_anchor_lens_null_lift.py (4) — load-bearing L1.2
    invariant `anchor_lens_byte_identity_null_lift`: full public
    cognition lane byte-identical for surface, trace_hash, and
    aggregate metrics between anchor_lens_id=None and
    "default_unanchored_v1".
  - tests/test_anchor_lens_pack_seam.py (9) — AST refuses any
    `packs.anchor_lens` import from truth-path modules (cognition /
    trace / pipeline / intent / propagation / vault / algebra) AND
    refuses any truth-path import from the loader itself.

Lane evidence
  - All 37 anchor-lens tests pass.
  - python -m core.cli eval cognition → public 100/100/91.7/100
    byte-identical (lens loaded but no composer reads it).
  - core demo register-tour --json → all_claims_supported: True
    (R5 seam still holds; L1.2 doesn't perturb register).
  - Full lane: 2669 passed / 4 skipped / 1 pre-existing failure
    (+37 over L1.1's 2632; the one failure remains
    test_all_preamble_explains_combined_run, unrelated).

Trust boundaries (per CLAUDE.md / ADR-0051)
  - safe_pack_id path-traversal rejection at loader entry.
  - No dynamic imports.
  - Loader is read-only; mutation only via ratify script.
  - Seam test refuses any new anchor-lens import upstream of the
    realizer.  L1.3 will widen the allow-list to include composer
    files at the same time it adds composer behaviour — exactly the
    way the register seam was widened at R2.

What L1.2 deliberately does NOT do
  - No composer consumes the lens (that's L1.3).
  - No TurnEvent / ChatResponse telemetry fields (L1.4).
  - No `core chat --anchor-lens` CLI flag (L1.4).
  - No anchor-lens-tour demo (L1.4).
2026-05-19 19:46:34 -07:00
Shay
7f0bad3e20 feat(register): R5 — operator-visible register telemetry + tour demo
ADR-0072 ratified + implemented.  Closes the register subsystem
inside-out arc (R1 ADR-0068 → R5 ADR-0072): the presentation axis is
now operator-visible, CI-falsifiable, and audit-traceable.

Telemetry extension
  - TurnEvent + ChatResponse gain register_id + register_variant_id
    (12-char SHA-256 prefix of selected (opening, closing) pair;
    empty string for UNREGISTERED / no-decoration registers).
  - serialize_turn_event surfaces both fields in every audit JSONL
    line.  Pre-R5 callers stay byte-identical (defaults are "").

Decoration result widened
  - chat/register_variation.py: decorate_surface now returns
    DecorationResult(surface, opening, closing, variant_id).
  - decorate_surface_str alias preserves the pre-R5 string-only API
    for off-runtime callers.
  - chat/runtime.py updated at both call sites (stub + main).

Operator surface
  - core chat --register REGISTER_ID threads into
    RuntimeConfig.register_pack_id via _runtime_config_from_args.
  - Invalid id ⇒ RegisterPackError caught at cmd_chat and surfaced
    as a clean _die(...) before the REPL launches.

Narrative demo
  - evals/register_tour/run_tour.py walks 4 prompts × 3 ratified
    registers ({default_neutral_v1, terse_v1, convivial_v1}) and
    asserts three load-bearing seam claims:
      * all_grounding_sources_identical
      * all_trace_hashes_identical (ADR-0069 invariant C, falsifiable)
      * surfaces_vary_at_least_once (ADR-0071 seeded variation lift)
  - core demo register-tour exit code = 0 iff every claim holds.

Tests
  - tests/test_register_telemetry.py (6) — TurnEvent default,
    serializer keys, runtime emits register_id/variant_id for
    convivial/terse/unregistered, ChatResponse mirrors event fields.
  - tests/test_register_cli.py (7) — _runtime_config_from_args
    threading, invalid-id fail-fast, parser wires --register.
  - tests/test_register_tour_demo.py (7) — three seam claims pinned
    individually + all_claims_supported + per-cell register_id +
    variant_id discipline (empty for neutral/terse, non-empty for
    convivial).
  - tests/test_register_variation.py extended (4 new) — DecorationResult
    shape, decorate_surface_str alias, variant_id stability,
    bijection between non-trivial marker pairs and variant_ids.

Lane evidence
  - Full lane: 2632 passed / 4 skipped / 1 pre-existing failure
    (tests/test_cli_demo.py::test_all_preamble_explains_combined_run,
    unrelated to R5).
  - Cognition eval byte-identical: public 100 / 100 / 91.7 / 100.

Trust boundaries (per CLAUDE.md)
  - --register flag does not bypass ratification; loader validates the
    pack id through _find_pack and the ratify gate at load time.
  - variant_id is content-addressed; no raw markers leak into audit.
  - Telemetry stays redact-safe — register_id and variant_id are
    identifiers, not content, so include_content=False emits them
    unconditionally.
  - No new mutation surface; pack files on disk are not modified.
2026-05-19 19:03:07 -07:00
Shay
6207b5fd0e feat(register): R1–R4 register pack subsystem — deterministic surface variation
Introduces the presentation axis as a fourth pack class (sibling to identity /
safety / ethics), orthogonal to the truth path. Same input + same packs +
same register ⇒ bit-for-bit reproducible surface; varying any of the three ⇒
genuinely different output. No stochastic sampling.

ADR-0068 (R1): RegisterPack frozen dataclass, loader, ratify script, seam test.
  - default_neutral_v1 ratified as null register.

ADR-0069 (R2): realizer register parameter threaded through 9 composer entry
  points; RuntimeConfig.register_pack_id; three byte-identity invariants
  (A: None ≡ pre-R2 unregistered; B: None ≡ default_neutral_v1; C: trace_hash
  invariant under register). Amended to default-with-lint after 167-call-site
  scout: composers default to UNREGISTERED, AST lint enforces explicit
  register= at runtime call sites.

ADR-0070 (R3): terse_v1 register, first non-neutral pack. realizer_overrides
  schema with known-keys allow-list (disclosure_domain_count ∈ {1,2,3}).
  build_pack_surface_candidate reads override with fail-soft clamp. New
  invariant register_invariant_grounding asserts grounding_source +
  trace_hash byte-identical across {None, neutral, terse}.

ADR-0071 (R4): seeded surface variation via convivial_v1.
  chat/register_variation.py applies SHA-256-seeded marker selection from
  bounded discourse-marker buckets. ChatResponse.pre_decoration_surface routes
  truth-path surface to core/cognition/pipeline.py so trace_hash stays
  invariant under register (the load-bearing architectural fix — initially
  invariant C failed under convivial because decoration was leaking into
  trace_hash via response.surface). Empty-string marker entries now
  legitimate ("no marker this turn" is a valid seed pick). realizer_overrides
  schema widened with per_intent nested block (validated against IntentTag
  whitelist; wired but not exercised by convivial). Two new invariants:
  seeded_variation_replay_equivalence (fresh runtimes → byte-identical) and
  seeded_variation_turn_distinct (same prompt across turns → ≥2 distinct
  surfaces).

ADR-0072 (R5, draft): telemetry + operator surface — TurnEvent gains
  register_id and register_variant_id, core chat --register flag, core demo
  register-tour. Status: Proposed; not yet implemented.

Three ratified register packs ship: default_neutral_v1 (null), terse_v1
(disclosure_domain_count=1), convivial_v1 (3 openings × 3 closings).

Verification:
  - 84 register tests pass + 1 documented skip
  - Curated lanes green: smoke 67, cognition 120+1s, teaching 17, packs 6,
    runtime 19, algebra 132
  - Cognition eval byte-identical to pre-register baseline:
    public 100/100/91.7/100, holdout 100/100/83.3/100
  - Full lane: 2608 passed, 4 skipped, 1 failed (pre-existing
    test_cli_demo.py "Combined Demo" → "Run Every Demo" rename, unrelated)

Truth-path isolation: chat/register_variation.py is realizer-side; the seam
test (tests/test_register_pack_seam.py) refuses imports of packs.register
from intent classification, propagation, vault recall, trace hashing, and
algebra.
2026-05-19 16:52:36 -07:00
Shay
7af7892dd8 feat(intent+discourse): CompoundIntent + sub-plan composition
Adds compound-intent decomposition for prompts that ask multiple
things in one turn ("What is X, and why does it matter?",
"Explain X, but how does it work?", "What is X, and what is Y?").

Three landings in one PR (rule says additive; the three pieces
are inseparable for the runtime hook to do anything useful):

1. generate/intent.py
   * New ``CompoundIntent`` frozen dataclass — ordered tuple of
     ``DialogueIntent`` parts + raw_text + ``.primary`` back-compat
     accessor + ``.is_compound()`` helper.
   * New ``classify_compound_intent(prompt)`` sibling to
     ``classify_intent``.  Pure, deterministic, byte-stable.  Splits
     on closed connector list (``,\s+(and|but|because|while)\s+``);
     anaphoric tails ("why does it matter") get the prior part's
     subject substituted ("why does truth matter") then are
     classified independently.
   * ``classify_intent`` return shape is untouched — every existing
     caller still receives ``DialogueIntent``.
   * No new ``IntentTag`` introduced.  v1 semantic approximation:
     "why does X matter" routes to ``CAUSE(X)``; "matter" means
     causal/relevance support, not metaphysical importance.

2. generate/discourse_planner.py
   * New ``plan_compound_discourse(compound, mode, bundles)`` —
     concatenates per-part sub-plans in source order with a
     ``TRANSITION`` bridge (fact=None) between consecutive parts.
     No cross-part re-sorting.
   * New private kw-only ``_exclude_facts`` parameter on
     ``plan_discourse`` so subsequent sub-plans can avoid emitting
     the same facts the prior sub-plans already used (prevents
     "Truth is X. Truth is X." duplicates on shared-subject
     compounds).  Public signature ``(intent, mode, bundle)`` is
     unchanged.

3. chat/runtime.py
   * Helper ``_maybe_apply_discourse_planner`` now consults the
     compound classifier first.  When the prompt is multi-part it
     builds per-part bundles and calls ``plan_compound_discourse``;
     otherwise it follows the previous single-intent path.
   * Compound bypass: when upstream tagged the surface ``oov`` /
     ``none`` because the flat classifier saw a polluted subject
     (e.g. ``"truth, and why does it matter"``), but the compound
     decomposition reveals a pack-resident primary subject, the
     planner engages on the decomposed parts.  This narrowly widens
     the gate exclusively for compound prompts with substrate.
   * BRIEF mode upgrades to EXPLAIN for compound prompts —
     single-anchor sub-plans on shared subjects would emit duplicate
     anchor sentences in BRIEF.
   * Return shape widened to ``tuple[str, str] | None`` —
     ``(rendered_surface, new_source_tag)``.  ``new_source_tag`` is
     ``"teaching"`` when the plan uses any teaching fact, else
     ``"pack"`` — so downstream labels reflect actual provenance
     even on the compound bypass.  Both cold and warm call sites
     updated to apply both fields.

24 new tests pin: compound decomposition correctness, source-order
preservation across sub-plans, anaphoric-followup rewriting,
deterministic byte-stable plans, no new IntentTag introduced,
fact-dedup across sub-plans, compound-bypass engagement, and
source-tag correction on planner-engaged surfaces.

Lane re-measurement after 3 compound cases added to cases.jsonl
(24 total cases):

  flag off: articulate=0.0833, disclosure=0.1667, unarticulate=0.7500
  flag on : articulate=0.9167, disclosure=0.0000, unarticulate=0.0833

Note: disclosure flag-on dropped to 0.0 because the source-tag
correction now correctly labels compound-bypass surfaces as
``pack/teaching`` instead of letting the upstream ``oov`` label
inflate disclosure.  The two remaining unarticulate cases flag-on
are the walkthrough prompts targeted by the next landing.

Critical gates all green:
* flag off cognition byte-identical: public 100/100/91.7/100
* smoke suite 67/67
* 32/32 planner tests pass (helper + render + compound)
* 18/18 compound classifier tests pass
2026-05-19 12:23:58 -07:00
Shay
f03d7d04b3 refactor(runtime): collapse cold+warm planner hooks into one helper
Pre-cleanup before extending intent classification.  Extracts
``ChatRuntime._maybe_apply_discourse_planner(text, source_tag) ->
str | None`` and replaces the two duplicated blocks (cold-start
pack-grounded branch + warm post-walk branch) with single-line
``planned = ...; if planned is not None: assign`` call sites.

Signature locked: takes only the prompt and the already-classified
grounding source tag; returns the replacement surface or None.
Callers own assignment — the helper neither reads nor writes any
surface or articulation state.  The warm site additionally does the
``articulation = replace(articulation, surface=planned)`` follow-up
which the cold site does not need.

Gating discipline unchanged (re-pinned in 9 new tests):
* Returns None when ``self.config.discourse_planner`` is False.
* Returns None unless source_tag ∈ {"pack", "teaching"}.
* Returns None when the classified intent has no subject.
* Returns None on single-move plans (BRIEF mode / empty bundle).
* Returns None on empty rendered string.

Behavior is byte-identical to the pre-dedup state — same metrics:
  flag off: multi=0.1429, primed_multi=0.0000, conn=0.0769
  flag on : multi=0.5238, primed_multi=0.5000, conn=0.2308
cognition eval byte-identical: public 100/100/91.7/100.
smoke suite 67/67.

The two paths now cannot drift; the upcoming intent classifier
extension lifts both branches in lockstep.
2026-05-19 12:04:15 -07:00
Shay
2aae25f4e2 feat(runtime): engage discourse planner on cold pack/teaching path
Option 2 of the lane-isolation work.  Mirrors the existing warm-path
hook into the cold-start branch (``gate_decision.fire`` ⇒ stub
response): after ``_maybe_pack_grounded_surface`` succeeds and the
result is pack- or teaching-grounded, build the same DiscoursePlan
and replace ``pack_surface`` with the rendered plan whenever it has
more than one move.

This closes the gap option 1 exposed: cold-start one-shot prompts
("Tell me about truth.", "Describe wisdom.", "Give me an example of
truth.") now produce deterministic multi-clause output without any
priming setup — the planner becomes the spine for grounded surfaces,
not a warm-only sidecar.

Gating discipline preserved:
* Engages only when pack_source_tag in {"pack", "teaching"}.  Cases
  routed to vault, none, or the discovery-signal disclosure are
  untouched.
* BRIEF mode collapses to a single ANCHOR move which renders
  byte-equivalent to the existing pack-grounded composer, so
  flag-off cognition byte-identity is preserved.
* Empty bundles → empty plan → no surface change (planner is total).

A/B on multi_sentence_response (21 cases, public/v1):

  flag off: multi=0.1429, primed_multi=0.0000, conn=0.0769
  flag on : multi=0.5238, primed_multi=0.5000, conn=0.2308

Cold-start lift: multi +38pp, conn +15pp.  Primed metric unchanged
(those cases already engaged the warm hook in step 5).

Sample cold-start surfaces flag-on:
* "Tell me about truth."
  → "Truth is a claim or state grounded by evidence and coherent
    judgment. Furthermore, truth belongs to cognition.truth. In
    turn, truth grounds knowledge."
* "Describe wisdom."
  → "Wisdom is sound judgment informed by knowledge and experience.
    Furthermore, wisdom belongs to cognition.wisdom. In turn,
    wisdom orders judgment."
* "Give me an example of truth."
  → "Truth is a claim or state grounded by evidence and coherent
    judgment. In turn, truth grounds knowledge."  (EXAMPLE mode:
    anchor + relation, no support)

Gates all green:
* flag off: cognition eval byte-identical
  - public 100/100/91.7/100, holdout 100/100/83.3/100
* smoke suite 67/67
* conversational_thread_coherence: 3 unwanted placeholders flag off
  and flag on (no regression)
* 136/136 planner + grounding + intent + lane tests pass
2026-05-19 11:55:12 -07:00
Shay
30948a1605 feat(runtime): wire discourse planner behind RuntimeConfig flag
Step 5 of the discourse-planner sequencing.  Closes the chain:

    classify_intent + classify_response_mode
      -> grounding_bundle_for(subject)
      -> plan_discourse(intent, mode, bundle)
      -> render_plan(plan)
      -> response_surface

Adds RuntimeConfig.discourse_planner (default False).  When True, the
runtime — after the warm pack/teaching-grounded surface is set —
classifies the response mode, assembles a GroundingBundle from the
ADR-style accessors, builds a DiscoursePlan, and replaces the warm
surface with the deterministic multi-clause rendering whenever the
plan has more than one move.

Gating discipline:
* Engages only on warm_grounding_source in {"pack", "teaching"} so
  vault/none turns and the discovery-signal CAUSE/VERIFICATION
  disclosure are preserved exactly.
* BRIEF mode always collapses to a single ANCHOR move, so flag-on
  with BRIEF intent is byte-identical to flag-off.
* Empty bundles produce empty plans; the runtime falls through to
  the existing warm surface untouched.

Adds render_plan(plan) to generate/discourse_planner.py — a pure,
deterministic multi-clause renderer with fixed canonical connectives:
  ANCHOR    : capitalized opening sentence
  SUPPORT   : "Furthermore, ..."
  RELATION  : "In turn, ..."
  TRANSITION: "Consequently, ..."
  CLOSURE   : skipped when fact is None
Every visible token is a verbatim pack lexicon entry, gloss, or
reviewed teaching chain string — no synthesis.

13 new tests pin:
* render_plan empty/brief/paragraph shape
* canonical connectives present in paragraph rendering
* deterministic + verbatim-fact invariants
* RuntimeConfig.discourse_planner defaults False
* Flag-off surface has no planner connectives
* Flag-on lifts produce structurally well-formed multi-sentence
  output on grounded substrate

Lift measurement (multi_sentence_response public/v1, 15 cases):
* flag off: multi=0.40, connective=0.50, grounded=0.40
* flag on : multi=0.40, connective=0.60, grounded=0.40
  -> connective_present_rate +10pp; multi-sentence count flat
     because the existing narrative composer's literal "." chars in
     tags like "cognition.truth" already trigger sentence splits in
     the lane regex.  Real lift is form quality: e.g. "Tell me about
     truth" now renders as "Truth is a claim or state grounded by
     evidence and coherent judgment.  Furthermore, truth belongs to
     cognition.truth.  In turn, truth grounds knowledge." instead of
     the prior provenance-laden narrative surface.

Critical gates (all green):
* flag off: cognition eval byte-identical
  - public 100/100/91.7/100, holdout 100/100/83.3/100
* smoke suite 67/67
* conversational_thread_coherence: 3 unwanted placeholders flag off
  and flag on (no regression)
* planner JSON byte-stable across calls (contract tests)
* grounding source order preserved (sidecar tests)
2026-05-19 11:29:25 -07:00
Shay
e06fda5b8b feat(runtime+evals): warm-path pack grounding + three long-span lanes
Step 1 — warm_grounding_stability targeted patch
- chat/runtime.py:_maybe_pack_grounded_surface accepts allow_warm=True;
  warm path invokes it after articulation and overrides
  response_surface / articulation / grounding_source when pack-grounded
  or teaching-grounded.
- CAUSE / VERIFICATION without a teaching chain on warm path emits the
  unknown-domain disclosure (matches cold-path discovery-signal doctrine
  — no fabricated vault content).
- warmed_session_consistency public lane: warm_grounding_stability
  0.0 → 1.0, grounding_match_rate 1.0, telemetry_consistency 1.0.
- Cognition lane byte-identical (public 100/100/91.7/100, holdout
  100/100/83.3/100).  Full suite 2294 passed.

Step 2 — three new red eval lanes (measurement substrate)
- conversational_thread_coherence: 6 cases / 45 turns; per-turn
  no_placeholder / not_walk_fragment / length / is_grounded predicates
  + per-case topic_anchor and no_topic_drift.  Baseline: grounded
  0.93, topic_anchor 0.50, no_topic_drift 0.83.
- multi_sentence_response: 15 cases over Explain/Tell/Describe/Walk/
  Example/Essay shapes; predicates sentence_count >= 2, non-fragment,
  connective_present, subject_named.  Baseline: multi_sentence 0.53,
  connective 0.10 — biggest architectural gap.
- self_consistency_over_time: 7 cases; same probe at multiple turn
  indices with unrelated fillers interleaved.  Baseline: byte_identical
  0.86 (one CAUSE-no-chain disclosure drifts under accumulation).

All three lanes deterministic, lexical-predicate-only — no LLM judge,
no embedding similarity.  Red-on-creation by design.  See
notes/long_span_fluency_baseline_2026-05-19.md.
2026-05-19 08:26:38 -07:00
Shay
c6b4f1d21e fix(runtime): config-replace + thin API wrappers + stale docstring
Three independent hygiene fixes named in the 2026-05-19 design review.
All small, all observable, none architectural.

1. ``RuntimeConfig`` flag drop on pack_id / frame_pack override
   chat/runtime.py:306-320 used to enumerate fields by hand when
   reconstructing RuntimeConfig under the pack_id / frame_pack
   override path.  The list stopped at ``admissibility_margin`` and
   silently dropped FIVE newer flags: identity_pack, ethics_pack,
   forward_graph_constraint, composed_surface, thread_anaphora.
   Caller side-effect:

     ChatRuntime(pack_id="x", config=RuntimeConfig(composed_surface=True))
       .config.composed_surface == False  # silently lost

   Fix: ``dataclasses.replace(config, input_packs=..., frame_pack=...)``.
   Every field on the dataclass survives by construction; future
   additions never need a synchronized edit on this path.

2. Stale CAUSE / VERIFICATION docstring
   tests/test_intent_classification_extensions.py described a sixth
   runtime-side fix (pack_grounded_surface fallback for
   CAUSE/VERIFICATION) that was considered, reverted, and the file's
   own test classes pin the opposite contract.  Docstring now states
   the doctrine correctly: no fallback, deliberately, so the discovery
   layer can log the teaching-gap signal.

3. Thin convenience wrappers: respond / achat / arespond
   tests/test_achat.py and tests/test_language_pack_runtime.py
   referenced these public methods since 2026-05-14, but they were
   never implemented on ChatRuntime — those 12 tests had been red on
   every full-lane run since the rebase.  Added as thin wrappers:

     respond(text) -> ChatResponse.surface
     achat(text)   -> async wrapper around chat()
     arespond(text)-> async wrapper around respond()

   The async wrappers are deliberately NOT genuinely non-blocking —
   the underlying CPU-bound walk/recall/composition remains sync.
   Docstrings say so explicitly.  Callers needing real concurrency
   should wrap in asyncio.to_thread at the call site; promoting the
   wrappers to true async event-loop integration is a future change
   gated by an actual concurrent caller.

Regression coverage:
  tests/test_runtime_config_passthrough.py — 4 tests
    - all 19 RuntimeConfig fields survive a pack_id override
    - all five newer flags survive a frame_pack override
    - no-override path preserves caller config by identity (no rebuild)
    - the four public methods exist and are callable

Verification:
  44/44 affected tests green (was 12 red pre-fix).
  Cognition eval byte-identical on both splits.
  No surface-format change; this commit is pure plumbing.
2026-05-19 07:04:10 -07:00
Shay
b52e04a72f fix(intent): five conversational definition patterns + polarity-stopword
The 2026-05-19 cumulative live probe surfaced a stark gap: ~52% of
realistic conversational definition prompts ("Define X", "What does
X mean?", "What is to V?", "How does X work?", "What causes X?")
returned ``grounding_source="none"`` *even though every subject
lemma was pack-resident* across the 9 mounted English packs.

Root cause: the bottleneck was intent classification + subject
extraction, not lexicon coverage.  Five patterns either had no rule
or routed to an intent the runtime dispatcher couldn't handle.  The
fluency assessment at
``/Users/kaizenpro/.codex/worktrees/6533/core/notes/fluency_assessment_2026-05-19.md``
named these as Root Cause #1 ("public chat path does not use the
cognitive spine") and Root Cause #3 ("proposition graphs are too
thin").  This commit closes the surface-level half of that gap;
the deeper answer-plan layer (gloss propositions, P3 in the
assessment) is the next step.

Patterns fixed in ``generate/intent.py``:

  1. ``Define X``        — added ``^define\s+`` rule mapping to
                           DEFINITION (placed after ``^what is/are``
                           so multi-word DEFINITION patterns still
                           prefer the question form).
  2. ``What does X mean?`` — was matching TRANSITIVE_QUERY with
                            relation=``mean``.  Now re-routes to
                            DEFINITION inside ``classify_intent`` so
                            ``pack_grounded_surface`` fires on X.
                            Other transitive relations (precede,
                            ground, etc.) remain TRANSITIVE_QUERY.
  3. ``What is to V?``   — added infinitive-marker strip to
                           ``_normalize_subject`` for DEFINITION /
                           RECALL.  ``to`` is gated on intent tag so
                           it never strips a transfer preposition
                           from CAUSE / VERIFICATION.
  4. ``How does X work?`` — added ``_HOW_DOES_X_RE`` (third-person
                            mechanistic-cause).  Distinct from the
                            first-person PROCEDURE rule ("How do I
                            X?").  Verbs: work / function / operate /
                            happen / exist / behave / act / emerge.
  5. ``What causes X?``   — added causative-verb rule (causes /
                            triggers / enables / prevents / drives /
                            produces / induces / yields) routing to
                            CAUSE with X as subject.

Deliberate NON-fix: I considered adding a ``pack_grounded_surface``
fallback in the CAUSE / VERIFICATION dispatcher when no teaching
chain matches the subject.  Reverted on review — that masks the
"would_have_grounded" discovery-candidate signal the teaching
pipeline uses to identify teaching-content gaps (see
``tests/test_discovery_candidates``).  CAUSE on a pack-resident
lemma without a teaching chain stays ``grounding_source=='none'``
so the discovery layer can log the gap honestly.

``chat/pack_grounding.py``:
  Extended ``_CORRECTION_TOPIC_STOPWORDS`` to include polarity
  markers (no / yes / maybe / perhaps / hardly / indeed / surely /
  definitely).  Without this the CORRECTION composer would
  short-circuit on ``no`` from "No, my parent disagrees" and miss
  the topical lemma ``parent``.

Cumulative probe lift (44 realistic conversational prompts):
  BEFORE: pack=16  none=23  oov=4  teaching=1  (52% NONE)
  AFTER:  pack=37  none=2   oov=4  teaching=1   ( 5% NONE)

  The remaining 2 NONE responses are CAUSE-shaped prompts with no
  teaching chain — deliberately preserved as the discovery-gap
  signal described above.

Tests: tests/test_intent_classification_extensions.py — 23 new
tests covering each pattern + the lift invariant.

Verification:
  Cognition eval byte-identical on both splits (100/100/91.7/100
  public, 100/100/83.3/100 holdout).
  All 111 intent-affected tests green:
    test_intent_classification_extensions.py (23)
    test_intent_proposition_graph.py / test_intent_ratifier.py /
    test_intent_subject_extraction.py / test_narrative_example_intents.py
    test_procedure_surface.py
    test_correction_topic_lemma.py
    test_cross_pack_grounding.py (including the polarity-stopword fix)
    test_discovery_candidates.py
    test_contemplation_wiring.py
    test_en_core_polarity_v1_pack.py
2026-05-19 06:12:05 -07:00
Shay
bf8284fd47 Phase 2 — proposition-slot grounding for articulate_with_intent
Root cause: recalled_words was built from result.tokens (versor walk
neighbours) rather than the pack-resolved proposition slots. The walk
produces nearest-neighbour traversal artifacts; the proposition already
carries the correct subject/predicate/object from realize(). This made
ground_graph() fill <pending> obj slots with stop-word-adjacent tokens
instead of the actual answer content.

Fix — two changes, one new helper:

generate/intent_bridge.py
  • build_recalled_words_from_plan(plan, proposition, walk_tokens)
    Constructs the grounding tuple in priority order:
      1. plan.object  (ArticulationPlan — pack-resolved, already a word)
      2. proposition.object_  (Proposition — versor-decoded object slot)
      3. plan.predicate  (descriptive predicate word, richer than walk)
      4. plan.subject  (subject as last-resort semantic anchor)
      5. walk_tokens  (result.tokens alpha-filtered — supplemental backfill)
    Strips <pending>/<prior>/empty/non-alpha before deduplicating.
    Returns a deduplicated tuple in that priority order.
  • articulate_with_intent() gains an optional `proposition` param
    (typed as object to avoid import coupling at the call site).
    When provided, build_recalled_words_from_plan() is called to
    replace the raw recalled_words before ground_graph() runs.
    When omitted, behaviour is byte-identical to Phase 1 (backward
    compatible: all existing callers and tests pass unchanged).

chat/runtime.py
  • The single articulate_with_intent() call site now passes
    proposition=proposition so the bridge receives the full
    pack-resolved proposition for grounding. walk_tokens (the old
    recalled_words) are passed through as supplemental backfill.
  • No change to ChatResponse, TurnEvent, GenerationResult, or any
    ADR-gated schema.
2026-05-18 18:18:31 -07:00
Shay
d5a6e81b33 feat(adr-0067): cross-pack teaching chains — Plan Phase 4 closed
ADR-0064 bound each teaching corpus 1:1 to a single ratified pack;
chains whose subject + object resolved to different packs were
dropped at load time. Phases 1–3 ratified the per-pack DAGs needed
to lift that constraint safely.

ADR-0067 introduces a deliberately narrow cross-pack chain shape.
Each entry carries explicit subject_pack_id and object_pack_id
fields, and the loader verifies per-chain residency. Same-pack
entries are rejected as corpus-misfilings (anti-leakage). The
cross-pack composer is the fall-through after the in-pack composer,
so the cognition lane stays byte-identical.

Files:
- chat/cross_pack_grounding.py — CrossPackChain + loader +
  single-chain composer + multi-chain enumerators
- teaching/cross_pack_chains/cross_pack_chains_v1.jsonl — 5 seed
  chains (family×identity, parent×understanding, family×memory,
  identity×family, understanding×parent)
- chat/runtime.py — fall-through wiring in CAUSE/VERIFICATION
- chat/narrative_surface.py, chat/example_surface.py — merge
  cross-pack chains, per-chain pack-residency helpers
- tests/test_cross_pack_chains.py — 31 tests covering loader,
  surface, multi-chain access, runtime integration, in-pack
  precedence
- tests/test_narrative_example_intents.py — corpus-tag assertions
  widened to allow cross-pack aggregation

Verification:
- 31 new tests pass
- Curated lanes: smoke 67 / cognition 121 / teaching 17 / packs 6 /
  runtime 19 — all green
- Cognition eval byte-identical (public 100/100/91.7/100, holdout
  100/100/83.3/100)
- Full lane: 2098 passed, 2 skipped, 0 failed in 2:30
2026-05-18 17:22:43 -07:00
Shay
fe4cc2cd1f feat(adr-0066): session-thread context + opt-in anaphora prefix (Phase 3.1 + 3.2)
ADR-0066 P3.1 + P3.2.  Conversation now reads as a thread: turns
carry structured summaries of their predecessors and (optionally)
prefix new pack/teaching surfaces with deterministic backreferences.

P3.1 — chat/thread_context.py.

  TurnSummary(turn_index, intent_tag_name, subject, grounding_source,
              chain_id, corpus_id) — frozen, structured-fields-only.
  ThreadContext — bounded FIFO (default MAX_THREAD_TURNS=8) with
    snapshot(), recent_for_subject(), recent_subjects(), clear().
  recent_for_subject() excludes ungrounded tiers (oov/partial/none)
    by default — those are not strong-enough anchors.
  ChatRuntime.thread_context is owned at construction.
  _push_thread_summary runs at end-of-turn on BOTH stub and walk
    paths.  Teaching-grounded turns carry chain_id + corpus_id so
    downstream composers (P3.2) can detect same-chain reference.
  Cold-start intent classification now runs unconditionally (was:
    gated on sink attachment) so thread context captures subject
    regardless of sink state.

P3.2 — chat/anaphora.py.

  thread_anaphora_prefix(ctx, subject, intent_name, source) returns
  a deterministic prefix when:
    - current turn is pack/teaching tier
    - a prior pack/teaching turn on the same subject exists
    - the prior intent differs from the current intent

  Format (structural-fields-only — no prose):
    "(Recalling turn N: chain <chain_id>.) "    # prior was teaching
    "(Recalling turn N: <subject> grounded pack.) "  # prior was pack

  Opt-in via RuntimeConfig.thread_anaphora=False.  Default off keeps
  every existing surface byte-identical.

Live verification (with thread_anaphora=True + seeded context):
  > What is light?  # following a "Why does light exist?" teaching turn
  [pack] (Recalling turn 0: chain cause_light_reveals_truth.)
  light — pack-grounded (en_core_cognition_v1): cognition.illumination;
  logos.core; perception.clarity. No session evidence yet.

32 new tests passed.  Curated lanes green.  Cognition eval
byte-identical to pre-ADR baseline.
2026-05-18 17:01:34 -07:00
Shay
51aad0c2cd feat(adr-0065): OOV cliff → five-tier honesty gradient (Phase 2.1 + 2.2)
Replaces the flat "I don't know — insufficient grounding" disclosure
with a deterministic gradient that names specific vocabulary gaps
and gives operators concrete next steps.

P2.1 — OOV "teach me" surface (chat/oov_surface.py).

  When the intent classifier extracts a clean subject lemma but that
  lemma is not resident in any mounted lexicon pack, the runtime now
  emits a deterministic learning-invitation surface tagged
  ``grounding_source="oov"`` instead of the universal disclosure.

  Surface format (fixed template):

    "I haven't learned '{token}' yet (intent: {intent}).
     Mounted lexicon packs: {pack_list}.
     Teach me via a reviewed PackMutationProposal."

  The OOV token passes through ``core._safe_display.safe_display``
  before persistence — user-input sanitization at the trust boundary.
  No vocabulary is invented; no domain is inferred.  Honours the
  ADR-0027 proposal-only invariant: the surface invites a reviewed
  pack mutation, never silently mutates any pack.

  Refactored ``_maybe_pack_grounded_surface`` so every existing
  intent branch (COMPARISON / CAUSE / VERIFICATION / CORRECTION /
  PROCEDURE / DEFINITION+RECALL) falls through on a None composer
  result instead of early-returning.  The OOV invitation is the
  deterministic fall-through for any clean-subject prompt whose
  subject doesn't resolve.

P2.2 — Partial-grounding tier (chat/partial_surface.py).

  When exactly one of two COMPARISON lemmas resolves, the runtime
  emits a hedged surface that grounds the known side verbatim and
  disclaims the OOV side explicitly:

    "Whatever '{oov}' is, I can ground '{known}' — pack-grounded
     ({pack_id}): {d1}; {d2}.  I cannot ground the comparison
     without learning '{oov}' — teach me via a reviewed
     PackMutationProposal."

  Tagged ``grounding_source="partial"``.  Falls through to OOV
  invitation when both lemmas are OOV, and to full pack-grounded
  COMPARISON when both resolve — partial is the middle tier in the
  five-tier gradient.

  Also normalises trailing sentence punctuation on
  intent.secondary_subject at the COMPARISON boundary so prompts
  like "Compare A and B." (with the period) still resolve B
  correctly.

Five-tier gradient (vault → teaching → pack → partial → oov → none).

Test debt retired: four pre-existing tests asserted "OOV → universal
disclosure", which is exactly the contract P2.1/P2.2 inverted.
Rewritten to the new contract.  Plus test_procedure_surface.py
gained a test for the OOV gradient on procedure intents.

Verification:
  tests/test_oov_surface.py                       22 passed
  tests/test_partial_surface.py                   16 passed
  Cognition eval byte-identical:
    public  100% / 100% / 91.7% / 100%
    holdout 100% / 100% / 83.3% / 100%
  Curated lanes all green.
2026-05-18 16:41:45 -07:00
Shay
c492014815 feat(adr-0062): composed teaching-grounded surface (chain-of-chains)
Pre-ADR-0062, the teaching-grounded composer emitted exactly one
reviewed chain per surface — "light reveals truth" — even when the
corpus already contained an immediate follow-up "truth grounds
knowledge".  With 21 active chains after curriculum saturation v2,
many grounded prompts had a corpus-ratified follow-up the composer
silently dropped.

ADR-0062 adds the composed composer + an opt-in config flag:

  flag OFF (default):
    light — teaching-grounded (cognition_chains_v1): cognition.illumination;
    logos.core. light reveals truth (cognition.truth). No session evidence yet.

  flag ON:
    light — teaching-grounded (cognition_chains_v1): cognition.illumination;
    logos.core. light reveals truth (cognition.truth), which grounds
    knowledge (cognition.knowledge). No session evidence yet.

Follow-up resolution:
  - prefer cause; fall back to verification (deterministic preference)
  - cycle guard: 1-step cycles (A→B, B→A) blocked
  - pack-residency guard: follow-up's object must be pack-resident
  - bounded depth: v1 follows exactly one hop
  - degrades to single-chain BYTE-IDENTICALLY when no follow-up
    survives the guards (drop-in replacement)

Trust-boundary invariants preserved:
  - Every visible non-template token is lemma / pack-domain /
    humanize_predicate connective / template constant.  Only added
    template constant: ", which "
  - Deterministic: same chains → same surface bytes
  - Default-False flag pattern mirrors ADR-0047/0058
  - `versor_condition < 1e-6` invariant untouched (surface composition only)

Cognition lane null-drop invariant CI-pinned:
  Composed mode emits a strictly LONGER surface (extra follow-up
  clause); every expected_term passing flag-OFF must still pass flag-ON.
  Asserted in test_cognition_lane_metrics_unchanged_with_composed_flag
  for both public and holdout splits.  If a future change drops tokens,
  the test fails as a deliberate regression.

  public  flag OFF: intent 100% / surface 100% / term 91.7% / versor 100%
  public  flag ON : intent 100% / surface 100% / term 91.7% / versor 100% (identical)
  holdout flag OFF: intent 100% / surface 100% / term 83.3% / versor 100%
  holdout flag ON : intent 100% / surface 100% / term 83.3% / versor 100% (identical)

Live-prompt lift visible on ~12 of 21 active chains; the rest hit
cycle or pack-residency guards.  Saturation v2's clusters were
authored partly with composition in mind (thought→meaning→
understanding, inference→evidence→knowledge, etc.).

- core/config.py — `RuntimeConfig.composed_surface: bool = False`
- chat/teaching_grounding.py — `teaching_grounded_surface_composed`
  sibling to `teaching_grounded_surface`
- chat/runtime.py — dispatch branch in `_maybe_pack_grounded_surface`
  selects composed vs single-chain based on config flag
- tests/test_composed_surface.py — 11 tests pin: function-level
  (None on no chain / degrades when no follow-up / two-clause when
  follow-up exists / includes intermediate + final domains /
  deterministic / cycle guard / trust label preserved); runtime
  integration (default single-chain / flag-on composed / frozen
  config); cognition-lane null-drop invariant.

Lanes (regression): smoke 67 / cognition 121 / teaching 17 /
composed-surface 11 — all green.
2026-05-18 14:34:45 -07:00
Shay
bf7f7895fe feat(adr-0061): PROCEDURE intent routes to pack-grounded surface
Pre-ADR-0061 every "How do I X?" question fell through to the
universal disclosure even when X was a pack-resident lemma.  The
teaching corpus carries CAUSE/VERIFICATION chains only — procedural
knowledge is fundamentally different in kind from propositional
claims and deserves its own ratification path (deliberately out of
scope; a future parallel `procedure_chains_v1.jsonl` schema is
discussed in the ADR's non-goals).

ADR-0061 adds the honest cold-start fallback: ground the topic in
pack semantic_domains and note explicitly that ratified step-by-step
guidance does not exist yet.

Surface format:
  "procedure-grounded ({pack_id}): {lemma} ({d1}; {d2}).
   Step-by-step guidance for {lemma} is not yet ratified
   in this session."

Selector — **last** pack-resident lemma in the verb-phrase subject:
  "define a concept" → concept    (object beats verb)
  "verify a claim"   → verify     (verb wins when object is OOV)
  "correct an error" → correct
  "learn this"       → learn
  "do stuff"         → None       (falls through to universal disclosure)

Stopwords: only `be` and `have` (dialogue fillers).  Procedure verbs
are deliberately NOT stopworded so the verb-as-fallback rule fires
when the object is OOV — keeps surface coverage.

Trust-boundary invariants:
  - Every visible non-template token is lemma / pack-domain / template.
  - Deterministic: same subject_text → same bytes.
  - Returns None for fully-unknown utterances → universal disclosure
    fires.  Never fabricates surface from nothing (ADR-0053 contract).
  - "not yet ratified" trust-label preserved.

Cognition lane lift:
  public  : intent 100% / surface 100% / term 91.7% / versor 100%      (unchanged)
  holdout : intent 100% / surface 94.7%→100.0% / term 79.2%→83.3% / versor 100%

Two cases fixed:
  - procedure_define_010 ("How do I define a concept?") — surface +
    term `concept` now captured.
  - procedure_verify_034 ("How do I verify a claim?") — surface only
    (case has no expected_terms; the verb fallback grounds it).

Combined effect: holdout `surface_groundedness` closes to 100%; 4 of
5 architectural holdout misses now resolved (this ADR + ADR-0060 +
the supersede from epistemology v1).  Remaining 2 are UNKNOWN-intent
cases (unknown_spirit_041, unknown_word_018) — out of scope; deserve
their own ADR with distinct selector semantics.

- chat/pack_grounding.py — `_extract_procedure_topic_lemma` helper +
  `pack_grounded_procedure_surface` composer.
- chat/runtime.py — import + dispatch branch for `IntentTag.PROCEDURE`.
- tests/test_procedure_surface.py — 15 tests pin: extraction
  (last-wins / verb-by-elimination / be+have skipped / None on empty /
  strips punctuation / case-insensitive); surface (contains lemma /
  contains domains / pack_id / "not yet ratified" label / None for
  no-pack-lemma / deterministic); end-to-end through ChatRuntime.

Lanes (regression): smoke 67 / cognition 121 / teaching 17 /
procedure 15 — all green.

The non-negotiable field invariant (versor_condition < 1e-6) is
unaffected: this ADR changes surface composition only.
2026-05-18 14:22:19 -07:00
Shay
c9e858c266 feat(adr-0060): correction acknowledgement carries corrected-topic lemma
ADR-0053's cold-start CORRECTION surface was topic-blind: a user who
said "Actually, truth requires evidence" got a response referencing
`correction` but never `truth`.  The holdout case correction_truth_040
expected `term=['truth']` and missed — one of the architectural gaps
surfaced by the epistemology v1 curriculum unit.

ADR-0060 closes that gap by weaving the first pack-resident topical
lemma from the utterance into a fixed-template extension:

  correction received — pack-grounded ({pack_id}):
  {correction_domains}. Noted topic: {lemma} ({lemma_domains}).
  No prior turn in this session to correct yet.

Selection rule (deterministic, left-to-right token order):
  - skip stopwords: `correction`, `correct`, `be`, `have`
  - pick first pack-resident lemma
  - if none found → ADR-0053 topic-less template byte-identically

Trust-boundary invariants preserved:
  - Every visible non-template token is still lemma / pack-domain / template
  - Deterministic: same text → same bytes
  - Backward compatible: existing 15 ADR-0053 tests pass byte-identically
  - "No prior turn in this session to correct yet." trust label kept

Cognition lane lift:
  public  : intent 100% / surface 100% / term 91.7% / versor 100%   (unchanged)
  holdout : intent 100% / surface 94.7% / term 75.0%→79.2% / versor 100%

The +4.2pp matches the single-case fix exactly (correction_truth_040).
Remaining 3 holdout misses (procedure_define_010, unknown_spirit_041,
unknown_word_018) are out of scope for this ADR.

- chat/pack_grounding.py — `_extract_correction_topic_lemma` helper +
  optional `text` parameter on `pack_grounded_correction_surface`.
- chat/runtime.py — single-line call-site change to pass `text` through.
- tests/test_correction_topic_lemma.py — 14 new tests pin:
  extraction (first lemma / skips correction / skips fillers / None on
  empty / strips punctuation / case-insensitive); surface (contains
  corrected lemma / contains topic domains / degrades to ADR-0053
  byte-identically / preserves trust label / deterministic / correct
  pack_id); end-to-end (correction_truth_040 emits 'truth' / no-pack-
  lemma still grounds).

Why text-level extraction, not intent.subject:
  `intent.subject` after ADR-0049 head-noun extraction returns
  ", truth requires evidence" for the test prompt — the CORRECTION
  intent's subject-extractor preserves the post-marker tail.  Parsing
  the raw text at the surface layer is cleaner; isolates the fix;
  doesn't perturb upstream classification logic.

Lanes (regression): smoke 67 / cognition 121 / teaching 17 /
correction tests 29 (15 ADR-0053 backward-compat + 14 ADR-0060 new) —
all green.

The non-negotiable field invariant (versor_condition < 1e-6) is
unaffected: this ADR changes surface composition only.
2026-05-18 14:14:27 -07:00
Shay
29449f3775 feat(adr-0059): correction-pass telemetry emission — backward perturbation auditable
`ChatRuntime.correct()` propagates a backward perturbation through the
session graph (per session/correction.py): each past turn whose output
versor has non-trivial CGA-alignment with the correction versor is
blended toward it (decayed by graph distance).  The forward regen turn
that followed already emitted a TurnEvent — but the backward
perturbation itself was invisible to the telemetry sink.

ADR-0059 closes that gap with a discriminated event line.

- chat/telemetry.py — adds `serialize_correction_event` +
  `format_correction_event_jsonl` emitting one JSONL line discriminated
  by `"type": "correction"`.  Payload: target_turn, records_count,
  turns_skipped, turn_idxs_affected, max_delta_norm, mean_delta_norm,
  SHA-256 correction_versor_digest, pack ids.  No raw versor coordinates.
- chat/runtime.py — `_emit_correction_event` (mirrors
  `_emit_turn_event`); called from `correct()` after the graph state
  is updated but before the forward regen turn.  No-op without sink.
- tests/test_correction_telemetry.py — 7 tests pin: no-op without
  sink, emission with sink, payload shape (required keys + types +
  ranges), SHA-256 digest shape, trust boundary (no versor
  coordinates leaked), determinism (byte-identical lines across
  runs), correction event and turn event coexist in the sink.

Trust boundary (per CLAUDE.md):
  - Metadata-only: only L2 deltas + SHA-256 digest.
  - No implicit wall-clock.
  - Deterministic: same CorrectionResult → byte-identical line.
  - Sink contract unchanged: `emit(line: str)`.
  - `versor_condition < 1e-6` invariant: untouched (telemetry-only).

Verification: smoke 67 / runtime 19 / correction telemetry 7 — green.
2026-05-18 13:47:48 -07:00
Shay
db6ce08589 feat(adr-0056): wire contemplation into live turn path (opt-in)
ChatRuntime.attach_contemplation(enabled=True) flips an opt-in
flag; when on, each emitted DiscoveryCandidate runs through
teaching.contemplation.contemplate before the sink writes the
JSONL line.  Default off ⇒ Phase B raw output preserved byte-
identical.

Trust boundary
- Contemplation is read-only over pack + corpus.
- Without an attached discovery sink the flag is inert (no hidden
  work — emission requires an observable destination).
- Active teaching corpus on disk byte-identical pre/post.

Lanes: smoke 67 / runtime 19 / cognition 121 / contemplation-
wiring 6 — all green.  Cognition eval unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 10:13:44 -07:00
Shay
07d35c0f54 feat(adr-0055): Phase B — DiscoveryCandidate emission from turn loop
Lands the first deterministic trigger of the discovery → reviewed-
memory loop. Candidates are structured evidence; emission is
opt-in via attach_discovery_sink and NEVER mutates the active
teaching corpus.

- teaching/discovery.py: DiscoveryCandidate dataclass + pure
  extract_discovery_candidates(turn_event, intent, subject) rule
  firing. Phase B fires only the would_have_grounded trigger:
    grounding_source == "none"
    AND intent ∈ {CAUSE, VERIFICATION}
    AND subject lemma in ratified cognition pack
    AND (subject, intent) NOT in active corpus
  candidate_id = SHA-256 of canonical JSON payload — replay-stable.
  Other DiscoveryTrigger literals (successful_comparison,
  hedge_acknowledged, oov_resolved_via_decomp) are reserved for
  later phases.

- teaching/discovery_sink.py: DiscoveryCandidateSink protocol,
  DiscoveryBufferSink (in-memory), DiscoveryMonthlyFileSink
  (append-only JSONL, <root>/<YYYY>/<YYYY-MM>.jsonl rollover,
  injectable clock).

- chat/runtime.py: opt-in attach_discovery_sink, post-turn
  emission inside _stub_response only when caller threads
  classified intent forward (gate-fire fall-through site).
  Intent classification at the call site reuses the same
  deterministic classifier already invoked by
  _maybe_pack_grounded_surface for the empty-vault English path.

Trust boundary: candidates write to a separate sink/file path
only; the active corpus on disk is never touched. Tests
explicitly assert corpus bytes are byte-identical before and
after a candidate-emitting turn.

Tests: tests/test_discovery_candidates.py — 24 tests covering
pure-predicate rule firing, every short-circuit path,
deterministic candidate_id, sink opt-in, runtime parity with no
sink, monthly rollover semantics, append-only behaviour, no
corpus mutation.

Lanes: smoke 67, cognition 121, runtime 19, teaching 17, packs 6
— all green. Cognition eval metrics unchanged on dev / public /
holdout splits. versor_condition < 1e-6 invariant untouched.
2026-05-18 08:26:04 -07:00
Shay
e975faf8a8 feat(adr-0053): cognition lane closure — corpus expansion + CORRECTION acknowledgement
Closes both cognition splits at 100% surface_groundedness.  Three
parts:

1. Teaching corpus expansion (no code).  cognition_chains_v1.jsonl
   grows 3→10 chains.  3 close dev-split misses (correction,
   creation, light-as-VERIFICATION); 4 pre-empt the analogous
   holdout pattern (CAUSE/VERIFICATION on truth + wisdom).  Every
   subject/object is a pack lemma; every connective is a recognised
   humanize_predicate predicate.

2. CORRECTION acknowledgement branch.  New
   `pack_grounded_correction_surface()` in chat/pack_grounding.py,
   wired into `_maybe_pack_grounded_surface` for cold-start
   CORRECTION intents.  Fixed-template surface with distinct
   trailing disclosure ("No prior turn in this session to correct
   yet.") — distinguishes the cold-start acknowledgement from the
   DEFINITION-of-correction surface.  The post-correction reviewed-
   teaching path in teaching/correction.py is unchanged.

3. Diagnostic memory.  Saves the dev-split generalization finding:
   the ADR-0048→0052 chain is NOT overfit.  Public/dev gap was
   teaching-corpus content coverage, not architecture.

Eval deltas (both splits run, post-ADR-0053):
                       public   dev
  intent_accuracy        100%   100%   (=)
  surface_groundedness   100%   100%   SATURATED
  term_capture_rate    91.7%  78.6%
  versor_closure_rate    100%   100%   (=)

Public surface_groundedness: 92.3% → 100%   (+7.7 pp)
Dev    surface_groundedness: 69.2% → 100%   (+30.8 pp)

Tests: tests/test_pack_grounded_correction.py (15 new tests).
Lanes green: smoke (67), cognition (121), runtime (19),
teaching (17), packs (6).

Scope limits: holdouts (19 cases) not yet in the official
`core eval cognition` runner (--split accepts only {dev, public});
the CORRECTION surface does not yet echo the corrected-subject
lemma (relevant only for holdout case `correction_truth_040`).
2026-05-18 07:43:39 -07:00
Shay
c6ade6c76f feat(adr-0052): teaching-grounded CAUSE/VERIFICATION surface 2026-05-18 07:13:43 -07:00
Shay
ecd580479a feat(adr-0050): pack-grounded COMPARISON surface
Sibling to ADR-0048's DEFINITION/RECALL pack-grounded surface for
the COMPARISON intent.  `pack_grounded_comparison_surface(a, b)` in
`chat/pack_grounding.py` composes a deterministic side-by-side
surface from both lemmas' pack `semantic_domains`, joined by the
fixed connective "contrasts with":

  "{a} (d_a1; d_a2) contrasts with {b} (d_b1; d_b2) — pack-grounded
   ({pack_id}). No session evidence yet."

`chat/runtime.py:_maybe_pack_grounded_surface` gains a COMPARISON
branch that runs before the DEFINITION/RECALL check.  Engages only
when both `intent.subject` and `intent.secondary_subject` are pack
lemmas and differ (identical-lemma comparison defers to disclosure).
Order-sensitive by design — matches the graph-layer's directional
CONTRAST edge.

Cognition eval (13-case public split):
  surface_groundedness  61.5% → 69.2%  (+7.7 pp)
  term_capture_rate     50.0% → 58.3%  (+8.3 pp)
  intent_accuracy            100.0%        (=)
  versor_closure_rate        100.0%        (=)

Case lifted: comparison_memory_recall_030 ("Compare memory and
recall").  Remaining unlift cases (CAUSE×2, VERIFICATION×1,
CORRECTION×1) need teaching-store chains or operator-driven
inference — pack lookup cannot supply causal explanations,
verifications, or corrections without fabrication.

Tests: tests/test_pack_grounded_comparison.py (15 tests).
Lanes green: smoke (67), cognition (121), runtime (19), algebra
(132), teaching (17), packs (6).
2026-05-18 06:59:53 -07:00
Shay
c28e107dc7 feat(adr-0048): pack-grounded surface for cold-start DEFINITION/RECALL
Closes the surface-grounding gap isolated by ADR-0047's
characterisation.  Adds the ratified cognition pack as a second
grounding source alongside the session vault.

== chat/pack_grounding.py (new) ==

Loads en_core_cognition_v1's lexicon once (cached; immutable pack)
and exposes:

  pack_grounded_surface(lemma) -> str | None

Returns a deterministic, fully pack-sourced surface:

  "{lemma} — pack-grounded ({pack_id}): {d1}; {d2}; {d3}. No session evidence yet."

Every visible atom is the lemma or a verbatim semantic_domains
string from the pack.  No rewording, no synthesis, no LLM.

== chat/runtime.py ==

_stub_response gains optional pack_grounded_surface= parameter.
_maybe_pack_grounded_surface routes to the pack only when all four
hold: gate_source=="empty_vault", output_language=="en",
intent.tag in {DEFINITION, RECALL}, and intent.subject is a pack
lemma.  Safety/ethics refusal still takes priority above this branch.

ChatResponse and TurnEvent gain grounding_source ∈ {vault,pack,none}.
Main walk path tags responses "vault".

== core/cognition/pipeline.py ==

gate_fired detection moved from string equality on the universal
disclosure to provenance:

  gate_fired = response.vault_hits == 0 and response.grounding_source != "vault"

Same intent (suppress realizer template on gate-fired turns),
broader stub-path surface set.

== Characterisation (core eval cognition, 13-case public split) ==

  Metric                  Pre        Post     Δ
  intent_accuracy        100.0%     100.0%    0
  surface_groundedness    15.4%      46.2%   +30.8 pp
  term_capture_rate        0.0%      33.3%   +33.3 pp
  versor_closure_rate    100.0%     100.0%    0

Lift is non-uniform by design: only single-lemma DEFINITION/RECALL
on pack-known English subjects engage.  CAUSE/COMPARISON/VERIFICATION
and multi-word OOV subjects still return the universal disclosure —
fabricating those would violate the no-LLM-fallback doctrine.

== Tests ==

  tests/test_pack_grounding.py                          18 passed
  tests/test_semantic_realizer_integration.py (updated) 1 stub-path test
    pinned to the broader contract: surface is either universal
    disclosure or pack-grounded; never the realizer template.

== Lanes ==

  smoke 67  cognition 121  runtime 19  algebra 132
  teaching 17  packs 6

versor_condition(F) < 1e-6 invariant unaffected (no algebra changes).
2026-05-18 06:36:10 -07:00
Shay
f47a85a3e7 feat(adr-0047): wire forward graph constraint into the chat hot path
Closes ADR-0046's deferred follow-up: convert the PropositionGraph
into an AdmissibilityRegion BEFORE generate() runs on the live
chat path.

== generate/intent_bridge.py ==

New public helper:

    build_graph_from_input(text, plan) -> PropositionGraph

Same internal call as _build_graph_from_intent, without the
post-generation ground_graph step — suitable for forward use.

== chat/runtime.py ==

When the new flag is on and output language is English, build the
graph and the region before generate() and pass it via region=.
Empty / fully OOV graphs return AdmissibilityRegion(allowed_indices=None),
which generate() treats as unconstrained — the change is a true
no-op when the graph carries no in-vocab anchors.

== core/config.py ==

RuntimeConfig.forward_graph_constraint: bool = False

Default False preserves all pre-ADR-0046 behaviour and the ADR-0024
honest-refusal contract.  A first attempt wired the constraint
unconditionally; 15 tests failed with InnerLoopExhaustion because the
intent-derived graph's CGA neighbourhood doesn't intersect the walk's
candidate pool with top_k=8 on the current packs.  The honest answer
is not to widen top_k until the failure goes away nor to silently
relax — both erase the architectural information that the geometry
of the graph and the geometry of the walk are not yet co-located.
Opt-in preserves ADR-0024 and follows the ADR-0022→0026 transition-
window pattern.

== Characterisation (core eval cognition, 13-case public split) ==

A/B with the flag toggled:

  Metric                  OFF      ON      Δ
  intent_accuracy        100.0%   100.0%   0
  surface_groundedness    15.4%    15.4%   0
  term_capture_rate        0.0%     0.0%   0
  versor_closure_rate    100.0%   100.0%   0
  InnerLoopExhaustion       0        0     0
  non-trivial constraint   n/a    6 / 13   —

Findings:
- Wiring is correct and safe (no exhaustions, closure unchanged).
- Single-token in-vocab subjects engage the constraint
  (light/knowledge/meaning/memory/correction).
- Multi-word OOV subject phrases produced by the intent classifier
  fall through to unconstrained — this is the existing intent-
  classifier contract surfacing into geometry, not a constraint bug.
- Restricting which tokens the walk may visit did not change
  surface_groundedness or term_capture_rate on this lane.  The
  surface-grounding gap therefore lives downstream of propagation
  — in the realizer / surface-assembly / dialogue-role path — and is
  the next load-bearing pull.  This isolates the next ADR's scope.

== tests/test_forward_graph_constraint_wiring.py (5 tests) ==

  - DEFAULT_CONFIG.forward_graph_constraint is False
  - Default runtime answers without InnerLoopExhaustion
  - Opt-in runtime answers on a short benign input
  - Graph builder + build_graph_constraint produce a labelled
    AdmissibilityRegion ("graph:unconstrained" or "graph:<root_id>")
  - Flag is observable on the frozen RuntimeConfig

== docs/decisions/ ==

  - ADR-0047 ratifies the wire-up, opt-in rationale, and A/B numbers.
  - README index updated; the Pillar 1→2→3 section now reflects both
    the primitive (ADR-0046) and the live wiring (ADR-0047), and
    names the next pull (realizer / surface assembly) explicitly.

Verification (this branch):

  tests/test_forward_graph_constraint_wiring.py    5 passed
  tests/test_graph_constraint.py                   8 passed
  core test --suite smoke                         67 passed
  core test --suite cognition                    121 passed
  core test --suite runtime                       19 passed
  core test --suite algebra                      132 passed
  core test --suite teaching                      17 passed
  core test --suite packs                          6 passed
  core eval cognition                            metrics unchanged from main

versor_condition(F) < 1e-6 invariant unaffected.
2026-05-18 06:18:10 -07:00
Shay
226f14a941 feat(adr-0040): structured-logging sink for turn-event audit
Adds the canonical JSONL sink surface consuming TurnEvent records
that ADR-0039 made uniform across main and stub paths.  One
deterministic line per turn; redact-by-default trust boundary;
opt-in content emission; runtime auto-emits on attached sink.

Trust boundary (CLAUDE.md):
- Metadata-only by default — no surfaces or input tokens emitted.
  include_content=True opt-in at attachment time.
- Path fixed at construction for JsonlFileSink; no user-controlled
  paths interpreted at emit time.
- Sink errors propagate — telemetry failures should surface, not
  silently drop audit signal.

Determinism:
- sort_keys=True; compact separators. Same event → byte-identical line.
- No implicit wall-clock; timestamps caller-provided.
- Field set fixed; missing TurnEvent attrs fall back to safe defaults.

API:
- serialize_turn_event(event, **kwargs) -> dict  (pure)
- format_turn_event_jsonl(event, **kwargs) -> str (pure, deterministic)
- TurnEventSink Protocol; JsonlBufferSink; JsonlFileSink
- ChatRuntime.attach_telemetry_sink(sink, *, include_content=False)
- _emit_turn_event invoked after both turn_log.append sites

Wire format (alphabetised, always present): cycle_cost_total,
dialogue_role, ethics_pack_id, ethics_runtime_checkable_count,
ethics_upheld, ethics_violated, flagged, hedge_injected,
identity_pack_id, refusal_emitted, safety_pack_id,
safety_runtime_checkable_count, safety_upheld, safety_violated,
stub_path, turn, vault_hits, versor_condition.

Conditional: identity_* (when score present), surface /
walk_surface / articulation_surface / input_tokens (when
include_content=True), timestamp (when provided).

Files:
- chat/telemetry.py (new) — serializer, formatter, sinks
- chat/runtime.py — attach + emit + post-append calls
- tests/test_telemetry_sink.py (new) — 29 tests
- docs/decisions/ADR-0040-telemetry-sink.md (new)

Verification:
- Combined pack-layer + telemetry suite: 199 green (was 170 after
  ADR-0039; +29)
- CLI suites unchanged: smoke 67, runtime 19, cognition 121
- core eval cognition: intent 100%, versor_closure 100% (baseline)
2026-05-17 21:39:58 -07:00
Shay
f3cc408f82 feat(adr-0039): audit completeness — TurnVerdicts bundle, stub TurnEvent, hedge_injected
Closes three audit gaps left by the ADR-0035→ADR-0038 pack-layer
surface:

1. TurnVerdicts bundle (chat/verdicts.py) — frozen dataclass
   aggregating identity_score + safety_verdict + ethics_verdict +
   refusal_emitted + hedge_injected.  Attached to both
   ChatResponse.verdicts and TurnEvent.verdicts.  Fields typed as
   object for the same module-coupling reason as
   TurnEvent.safety_verdict.

2. Stub-path TurnEvent emission — _stub_response accepts optional
   tokens kwarg and appends a TurnEvent to turn_log when invoked
   from a real turn.  Audit consumers can now iterate turn_log
   end-to-end without missing stub paths.  Defensive call sites
   (correct() fallback) bypass the append by omitting tokens.

3. refusal_emitted / hedge_injected flags — runtime tracks whether
   it actually mutated the surface this turn.  hedge_injected uses
   idempotent-on-prefix semantics (True iff the runtime ADDED a
   hedge, not iff a hedge happens to be present).

Test-pattern note: previous "gate on rt.turn_log to detect main vs
stub" pattern is now broken; updated to gate on walk_surface ==
_UNKNOWN_DOMAIN_SURFACE.  One existing hedge-injection test gate
updated accordingly.

Back-compat: ADR-0035→0038 per-field accessors
(response.safety_verdict, etc.) still work.  New consumers should
read response.verdicts.

Files:
- chat/verdicts.py (new) — TurnVerdicts dataclass
- chat/runtime.py — _stub_response tokens kwarg + stub TurnEvent
  append + hedge_injected tracking + bundle construction
- core/physics/identity.py — TurnEvent.verdicts: object = None
- tests/test_turn_verdicts_bundle.py (new) — 16 tests
- tests/test_hedge_injection.py — gate fix for stub detection
- docs/decisions/ADR-0039-audit-completeness.md (new)

Verification:
- Combined pack-layer suite: 170 green (was 154 after ADR-0038)
- CLI suites unchanged: smoke 67, runtime 19, cognition 121
- core eval cognition: intent 100%, versor_closure 100% (baseline)
2026-05-17 21:32:46 -07:00
Shay
ad8495d777 feat(adr-0037,adr-0038): per-predicate ethics refusal + hedge injection
Two sibling escalation tiers above the audit-only ethics baseline,
both opt-in per commitment via the ethics pack JSON.

ADR-0037 — refusal_commitments

- EthicsPack.refusal_commitments (frozenset[str]; subset of
  commitment_ids; validated at load time, unknown id rejected)
- Generic refusal prefix: "I cannot proceed — boundary violated: "
- Source-tagged refusal ids: "safety:<id>" / "ethics:<id>"
- build_refusal_surface now takes (safety_verdict, ethics_verdict,
  ethics_pack); ADR-0036 single-arg call remains valid back-compat
- Default pack ships refusal_commitments: [] — audit-only floor
  preserved
- Re-ratified default pack (mastery sha changes with schema field)

ADR-0038 — hedge_commitments

- EthicsPack.hedge_commitments (sibling field; same validator)
- Mutually exclusive with refusal_commitments at load time
- Runtime prepends manifold's preferred_hedge_soft (fallback
  preferred_hedge_strong) when an opted-in commitment fires
  runtime-checkable
- Refusal supersedes hedge globally; stub path skips hedge (already
  a disclosure surface); main path only
- Idempotent on prefix (case-insensitive) — defends against
  ADR-0028 assembler hedges
- Does NOT flip _last_refusal_was_typed — hedge is not refusal

Surface contract:
- ChatResponse.walk_surface + articulation_surface preserved unchanged
  on both refusal and hedge paths (same audit discipline as ADR-0036)
- Only user-facing ChatResponse.surface (and TurnEvent.surface on
  main path) is mutated

Files:
- packs/ethics/loader.py — refusal_commitments + hedge_commitments
  fields; _validate_opt_in_subset; mutual-exclusion check
- packs/ethics/default_general_ethics_v1.json — both opt-in lists
  empty; re-ratified
- chat/refusal.py — generic prefix, source-tagged ids,
  violated_runtime_checkable_ethics, should_inject_hedge,
  build_hedge_prefix, inject_hedge
- chat/runtime.py — passes ethics_verdict + ethics_pack to refusal
  builder; hedge injection branch after refusal check
- tests/test_ethics_refusal_opt_in.py (new) — 16 tests
- tests/test_hedge_injection.py (new) — 22 tests
- docs/decisions/ADR-0037-per-predicate-ethics-refusal.md (new)
- docs/decisions/ADR-0038-hedge-injection.md (new)

Verification:
- Combined pack-layer suite: 154 green (was 116 after ADR-0036)
- CLI suites unchanged: smoke 67, runtime 19, cognition 121
- core eval cognition: intent 100%, versor_closure 100% (baseline)
2026-05-17 21:23:28 -07:00
Shay
a0372c951f feat(adr-0036): safety-only typed refusal policy
Runtime-checkable SafetyVerdict violations now replace
ChatResponse.surface (and TurnEvent.surface on the main path) with a
deterministic typed refusal string.  Ethics violations remain
audit-only.

Why safety-only: safety is the universal floor (ADR-0029,
never-swappable, fail-closed).  Ethics is swappable per-deployment;
wiring ethics into refusal would let pack-swappers silently change
refusal behavior via JSON edit.  Wrong coupling.

Why typed refusal (not hedge injection / not re-articulation): typed
refusal is deterministic, audit-detectable by prefix, and preserves
replayability.  Hedge injection would blur surface-preferences-driven
hedging vs predicate-driven refusal.  Re-articulation retry yields the
same surface (planner is deterministic; no refusal-bias hint surface
exists).  Deferred to a future ADR.

Refusal contract:
- ChatResponse.surface = typed refusal string
- walk_surface + articulation_surface = unchanged (audit preserved)
- runtime._last_refusal_was_typed = True (next-turn evidence for
  no_silent_correction)
- Only runtime_checkable=True violations refuse
- Stub path symmetric

Files:
- chat/refusal.py (new) — pure refusal builder + audit helpers
- chat/runtime.py — invoke build_refusal_surface after safety_verdict
- tests/test_safety_refusal.py (new) — 20 tests
- docs/decisions/ADR-0036-safety-refusal-policy.md (new)

Verification:
- 20 new tests; combined pack-layer suite 116 green
- CLI suites unchanged: smoke 67, runtime 19, cognition 121
- core eval cognition: intent 100%, versor_closure 100% (baseline)
2026-05-17 21:10:52 -07:00
Shay
514ace0cbf feat(adr-0035): turn-loop auto-invocation — surfacing only
Wires SafetyCheck and EthicsCheck into ChatRuntime at end-of-turn on
both the main articulation path and _stub_response.  Verdicts attach
to ChatResponse.safety_verdict / .ethics_verdict and TurnEvent.
Observational at v1: no refusal, no re-articulation, no behavioral
change.  Refusal policy is the next ADR with real verdict data in hand.

Runtime-checkable predicates today:
  - preserve_versor_closure         (via _FieldStateWithVersor adapter)
  - no_identity_override            (manifold hash before vs after; equal by construction)
  - no_silent_correction            (runtime._last_refusal_was_typed bookkeeping)
  - acknowledge_uncertainty         (IdentityScore.alignment + hedge detection)
  - disclose_limitations            (walk_surface == _UNKNOWN_DOMAIN_SURFACE)

Predicates with no runtime evidence (no_manipulation, no_fabricated_source,
defer_high_stakes_to_human_review, respect_user_autonomy, no_hot_path_repair)
honestly report runtime_checkable=False per the ADR-0032/0034 discipline.
They become checkable as classifiers and pipelines land — surface contract
doesn't change.

Test coverage: 14 new tests; combined pack-layer surface suite (loaders +
checks + turn-loop) now 122 green.  CLI suites unaffected: smoke 67,
cognition 121, teaching 17, runtime 19.  Cognition eval baseline preserved.
2026-05-17 20:57:33 -07:00
Shay
db5bc028f9 feat(adr-0034): EthicsCheck — structural surface parallel to SafetyCheck
Completes the predicate-surface layer for ethics packs, sibling to
ADR-0032's SafetyCheck.  Same registry-of-predicates shape; same
observational discipline; same honest reporting of runtime-checkable=False
for structural commitments that cannot be evaluated from per-turn evidence.

Five default predicates for the v1 commitments:

  acknowledge_uncertainty           — alignment < threshold ⇒ requires hedge
  defer_high_stakes_to_human_review — high_stakes ⇒ requires recommend_review
  disclose_limitations              — ungrounded ⇒ requires disclosure marker
  no_manipulation                   — structural; runtime_checkable=False
  respect_user_autonomy             — prescriptive ⇒ requires ≥2 options surfaced

`no_manipulation` is the ethics-side analogue of `no_hot_path_repair`
in SafetyCheck — an aggregate property enforced by realizer design and
review, not a per-turn metric.  Honest reporting rather than a silent
upheld pass.

ChatRuntime exposes `runtime.ethics_check`; turn loop does not
auto-invoke.  Refusal / re-articulation wiring is a future ADR.

Test coverage: 27 new tests; combined pack-layer surface suite
(identity + safety + ethics, loaders + checks) is now 108 tests, all
green.  Cognition (121), teaching (17), runtime (19), smoke (67)
unaffected.
2026-05-17 20:46:34 -07:00
Shay
dab7b9c061 feat(adr-0033): ethics packs — third pack-layer sibling to identity + safety
Completes the three-layer pack architecture:
  identity (who CORE is)  + safety (universal red lines)
                          + ethics (deployment-specific propositional commitments)

  manifold.boundary_ids = identity.boundary_ids
                        ∪ safety.boundary_ids
                        ∪ ethics.commitment_ids

Ethics packs are swappable like identity (fall back to default on load
failure) but propositional like safety (commitment ids union into the
manifold).  EthicsPackError inherits from ValueError; only when both
the requested and default packs fail does startup refuse.

Ships default_general_ethics_v1 with five commitments:
  - acknowledge_uncertainty
  - defer_high_stakes_to_human_review
  - disclose_limitations
  - no_manipulation
  - respect_user_autonomy

Ratified through identity_anchor template at SHA 81fc9b61c828….

Test coverage: 20 new tests; combined identity/safety/ethics surface
suite is 81 tests, all green.  Cognition (121), teaching (17), runtime
(19), smoke (67), and cognition eval all unaffected.
2026-05-17 20:41:04 -07:00
Shay
6f67e9a616 feat(safety): ADR-0032 — SafetyCheck structural surface
Closes the 'boundaries are checked at scattered call sites' gap noted
in ADR-0029.  Adds a centralized observational surface parallel in
shape to IdentityCheck — produces a verdict, does not refuse.  Wiring
verdicts into refusal paths is a future ADR.

Shape (parallel to IdentityCheck, different in mechanism):

  SafetyContext     — duck-typed input bag (field_state, citations,
                       refusal-was-typed flag, identity manifold hashes
                       before/after).  Every field optional with safe
                       defaults; absence of evidence is not evidence of
                       violation.
  SafetyCheckResult — per-boundary: boundary_id, upheld, reason,
                       runtime_checkable, evidence tuple.
  SafetyVerdict     — aggregate: pack_id, results (lex order on
                       boundary_id), upheld, violated_boundaries,
                       runtime_checkable_count.
  SafetyCheck       — registry of predicates; check(ctx, pack) returns
                       SafetyVerdict.  register(boundary_id, predicate)
                       adds custom predicates.

Five default predicates for v1 boundaries:

  preserve_versor_closure   runtime_checkable=True   field.versor_condition < 1e-6
  no_fabricated_source      runtime_checkable=True*  cited ⊆ allowed
  no_silent_correction      runtime_checkable=True   last refusal was typed
  no_identity_override      runtime_checkable=True*  hash before == hash after
  no_hot_path_repair        runtime_checkable=FALSE  code-path; static-analysis

  *Conditional on the caller supplying the necessary fields.

The honest answer on no_hot_path_repair: it is a code-path boundary
enforced by static analysis + code review.  Runtime cannot judge it.
A predicate that silently reported upheld=True would be a small lie —
exactly the kind of thing CLAUDE.md forbids.  SafetyCheck reports
runtime_checkable=False with a clear reason so auditors see the truth.

ChatRuntime integration:
  ChatRuntime.__init__ now constructs self.safety_check = SafetyCheck()
  alongside self._identity_check.  Turn loop does NOT auto-invoke at
  v1 — operators and future ADRs decide when/where to call it.

Files:
  packs/safety/check.py            new — SafetyCheck + value types +
                                   default predicates
  packs/safety/__init__.py         re-exports the new public surface
  chat/runtime.py                  constructs self.safety_check
  tests/test_safety_check.py       new — 20 tests covering each
                                   default predicate (positive +
                                   negative), unknown-boundary
                                   fallback, custom registration,
                                   defensive boundary-id rebinding,
                                   verdict aggregation, ChatRuntime
                                   integration
  docs/decisions/ADR-0032-safety-check-surface.md  Accepted
  docs/safety_packs.md             §SafetyCheck section added,
                                   known-limit #1 struck through
  memory/safety-pack.md            refreshed; new follow-up about
                                   turn-loop auto-invocation

Suite status (all green):
  cognition 121, teaching 17, runtime 19, formation 182, smoke 67
  identity / safety / surface divergence suites: 108 tests passing
  (was 88 before this ADR; +20 safety-check tests)

Scope limits (documented):
  - No auto-invocation in the turn loop.
  - No refusal wiring on violation.
  - No refactoring of existing scattered enforcement sites.
  - Defensive boundary-id rebinding masks predicate bugs; debug-mode
    surfacing is a future enhancement.
2026-05-17 20:25:22 -07:00
Shay
07ad3af845 feat(surface): ADR-0031 — score-decomposition surface (per-axis hedges)
Closes the 'identity hedges are generic' gap.  When IdentityCheck reports
that a specific axis is deviating AND the pack supplies an axis_hedges
entry for that axis, the assembler uses that axis's phrase instead of
ADR-0028's generic preferred_hedge_*.  The hedge text now names what is
actually at issue.

Selection: lex-smallest axis_id in (ctx.deviation_axes ∩ axis_hedges).
Deterministic; loader emits axis_hedges in lex order on axis_id.

Example surface at alignment=0.30 (strong band) under default pack:
  No deviation             → 'It seems that truth reveals reality.'
  truthfulness deviates    → 'Evidence is thin that truth reveals reality.'
  coherence deviates       → 'This does not yet cohere: truth reveals reality.'
  reverence deviates       → 'Reports suggest truth reveals reality.'

Same trajectory + truthfulness deviation, three different packs:
  default_general_v1   → 'Evidence is thin that truth reveals reality.'
  precision_first_v1   → 'The evidence does not support that truth reveals reality.'
  generosity_first_v1  → 'Truth reveals reality.'  (above generosity's strong=0.20)

Schema (additive, optional):
  surface_preferences.axis_hedges = {
    <axis_id>: { 'strong': str, 'soft': str, 'qualifier': str },
    ...
  }

Bounds: each phrase length 1–64; axis_id non-empty.  Absent block →
ADR-0028 byte-for-byte fallback.  Loader emits pairs in lex order on
axis_id for hashability + deterministic tie-break.

Files:
  core/physics/identity.py
    + class AxisHedge (frozen: strong, soft, qualifier)
    SurfacePreferences gains axis_hedges: Tuple = ()
  packs/identity/loader.py
    + _build_axis_hedges(): parse + bounds-check + emit lex-ordered tuple
  generate/surface.py
    SurfaceContext gains deviation_axes: frozenset[str] + axis_hedges tuple
    + _axis_specific_phrase(ctx): lex-smallest match or None
    _apply_hedge consults axis-specific phrase before ADR-0028 fallback
    Depth languages (he, grc) unchanged — ADR-0030 canonical phrases
  chat/runtime.py
    _build_surface_context lifts identity_score.deviation_axes and
    prefs.axis_hedges into SurfaceContext
  packs/identity/*.json
    Three v1 packs gain axis_hedges blocks (truthfulness, coherence,
    reverence — each pack uses voice consistent with its character)
  scripts/ratify_identity_packs.py (no change — idempotent)
  packs/identity/*.mastery_report.json
    Auto-refreshed.  New SHAs:
      default_general_v1   → 2ab7d469013509ba5030313ca9a609a443d0716e3ddcc5596f59858ce054f5d3
      precision_first_v1   → 78aa1e6a68a35c2c8576b6196a52d421b94f6d11e006128986902a4fd08679af
      generosity_first_v1  → 511f1ce20edd4266239da61443bfc93473a5433f20bfee6692a25a03073dc933

Tests: tests/test_identity_score_decomposition.py — 17 new tests:
  per-axis phrase selection, band gating still applies, pack swap with
  same deviation produces three different phrases, lex tie-break is
  deterministic, depth-language fallback to ADR-0030, backward compat
  with empty deviation_axes, and the contract that all three v1 packs
  ship axis_hedges for all three default-pack axes.

Suite status (all green):
  cognition 121, teaching 17, runtime 19, formation 182, smoke 67
  identity+safety+English+depth divergence 71
  score decomposition 17

Scope limits (documented in ADR-0031):
  - English-only at v1 (depth languages use canonical ADR-0030 phrases)
  - Lex tie-break is operational not semantic — pack authors can re-key
    if they need a different priority
  - No dominance-driven phrasing (Interpretation A); preserved as
    forward-compatible follow-up

Docs: ADR-0031 (Accepted) recorded; docs/identity_packs.md gains
§Axis-specific hedge phrases section and updated v1-pack SHAs; memory
'identity-packs.md' refreshed.
2026-05-17 20:16:22 -07:00
Shay
ece73c76d5 feat(safety): ADR-0029 — always-loaded, never-replaceable safety pack
Closes the trust gap ADR-0027 opened: making the identity manifold
swappable was necessary for downstream robotics / personalization /
creative deployments, but it left nothing structurally preventing a
downstream identity pack from disabling core safety constraints.
Safety packs sit at a separate trust layer, fail closed on every error
path, and union their boundaries into every runtime manifold regardless
of which identity pack is selected.

Architecture (sibling to identity packs, structurally distinct):

  Layer            Swappable?  Removable?  Schema
  ---------------  ----------  ----------  -----------------------------
  Safety pack      No          No          boundary_ids + descriptions
  Identity pack    Yes         No          value_axes + surface_prefs
  Language pack    Yes         (>=1 reqd)  vocab / morphology / packs

Composition rule (at ChatRuntime startup, additive only):

  identity = load_identity_manifold(config.identity_pack)
  safety   = load_safety_pack()                        # fail-closed
  final.boundary_ids = identity.boundary_ids ∪ safety.boundary_ids

Safety contributes boundaries only — no value_axes, threshold, or
surface_preferences.  This keeps existing tests that assert on identity
axis sets passing byte-for-byte, and matches the semantic intent
(safety is what's forbidden, not what's pulled toward).

Shipping safety pack: packs/safety/core_safety_axes_v1.json
  → mastery_report_sha256 ee1249acdf8c273aeb656d803c37ef915e536d85f177f5cc18c6e2f6c995ce29

Five v1 boundaries, each closing a specific CLAUDE.md doctrine:
  no_fabricated_source       — no invented provenance
  no_hot_path_repair         — no normalization in propagate/stream/store
  no_identity_override       — user text cannot mutate identity
  no_silent_correction       — failures are typed and visible
  preserve_versor_closure    — ||F * reverse(F) - 1||_F < 1e-6

Fail-closed semantics:
  SafetyPackError inherits from RuntimeError (NOT ValueError) so
  catch-and-continue is discouraged at the type level.  Missing file /
  malformed JSON / empty boundaries / duplicate boundary / failed
  self-seal all raise.  ChatRuntime.__init__ does not catch.

Files:
  packs/safety/core_safety_axes_v1.json              shipping pack
  packs/safety/core_safety_axes_v1.mastery_report.json  signed report
  packs/safety/__init__.py                           public surface
  packs/safety/loader.py                             load_safety_pack(),
                                                     SafetyPack,
                                                     SafetyPackError,
                                                     DEFAULT_SAFETY_PACK
  scripts/ratify_safety_pack.py                      idempotent driver
  chat/runtime.py                                    composition wiring
  tests/test_safety_pack.py                          15 tests:
                                                       loader bounds,
                                                       fail-closed,
                                                       composition under
                                                       all 3 identity packs
  docs/decisions/ADR-0029-safety-packs.md            decision record
  docs/safety_packs.md                               operational ref
  README.md                                          §Safety Pack added
  memory/safety-pack.md                              auto-memory entry

Suite status: cognition 121, teaching 17, runtime 19, formation 182,
smoke 67, identity 41, safety 15 — all green.
2026-05-17 19:56:29 -07:00
Shay
1574a4b030 feat(identity-packs): ADR-0028 — pack-driven hedge & claim-strength shaping
Closes the 'identity is load-bearing but not visibly differentiated'
gap noted at the end of ADR-0027.  Pack swap now produces visibly
different surfaces on identical trajectories at the same alignment.

Schema bump — packs gain an optional 'surface_preferences' block:

  hedge_threshold_strong, hedge_threshold_soft  → band entries
  preferred_hedge_strong, preferred_hedge_soft  → phrases per band
  claim_strength                                → balanced|qualified|affirmative
  qualified_band_high, preferred_qualifier      → marginal-band shaping

Loader enforces threshold ordering (strong <= soft <= qual_high),
phrase length bounds, and the enum-of-three for claim_strength.
Missing block resolves to defaults that reproduce pre-ADR behavior
byte-for-byte; existing tests pass unchanged.

Algorithm (deterministic, surface-only, no sampling/repair/normalize):

  alignment < strong              → preferred_hedge_strong + lower-cased surface
  alignment < soft                → preferred_hedge_soft + lower-cased surface
  soft <= alignment < qual_high
    and claim_strength=qualified  → preferred_qualifier + lower-cased surface
  otherwise                       → bare surface

Three v1 pack profiles:

  default_general_v1   balanced; 0.40 / 0.50 / 0.75 ; 'It seems that' / 'Perhaps'
  precision_first_v1   qualified; 0.55 / 0.70 / 0.85 ; 'Arguably,' / 'In some cases,' / 'Under certain conditions,'
  generosity_first_v1  affirmative; 0.20 / 0.30 / 0.50 ; default hedge phrases

Re-ratified.  New MasteryReport SHAs (superseding Phase-5):

  default_general_v1   → ddc1ba127231272660e6a435e177227558461b0278572a95635b416c3e1dec5a
  precision_first_v1   → cb5fb2323214a26afda33f2a67e22f38fe49f4763829d48ef67fd41241aba33c
  generosity_first_v1  → 94f2f49e1b16c7498fb52b8f9864eecc198618933dc8381a01b809c146826db7

Files touched:

* core/physics/identity.py — new SurfacePreferences dataclass;
  IdentityManifold gains 'surface_preferences' field with defaults.
* packs/identity/loader.py — _build_surface_preferences() parses,
  bounds-checks (threshold ordering, claim_strength enum, phrase
  length, threshold ranges); SurfacePreferences round-trips.
* generate/surface.py — SurfaceContext gains 7 new fields with defaults
  matching the pre-ADR module-level HEDGE_STRONG_THRESHOLD /
  HEDGE_SOFT_THRESHOLD; _apply_hedge takes the full context and
  implements the four-band algorithm; module-level constants retained
  for back-compat.
* chat/runtime.py — _build_surface_context lifts manifold.surface_preferences
  into SurfaceContext.
* packs/identity/*.json — three v1 packs gain surface_preferences blocks
  tuned to their roles; re-ratified via scripts/ratify_identity_packs.py
  (idempotent).
* tests/test_identity_surface_divergence.py — 15 tests covering hedge
  bands, claim_strength bands, pack-swap divergence proof, and runtime
  context wiring.

Suite status: cognition 121, teaching 17, runtime 19, formation 182,
smoke 67 — all green.  test_identity_packs.py 23/23, new
test_identity_surface_divergence.py 15/15.

Docs: ADR-0028 (Accepted) records the decision and verification; ADR-0027
status updated to point to ADR-0028 for deep realizer wiring; README
§Identity Packs notes the visible divergence; docs/identity_packs.md
gains a §Surface preferences section and closes the known-limit #1
about invisible surface differentiation.
2026-05-17 19:42:54 -07:00
Shay
c3e36f07b2 feat(identity-packs): ADR-0027 Phase 5 — ratify all three v1 packs
Drives the three v1 identity packs through the full formation pipeline
(Forge -> Compose -> Compile -> Run -> Ratify) and embeds the resulting
self-sealed MasteryReport SHAs into each pack file.  Companion
'<pack_id>.mastery_report.json' artifacts ship alongside.  Loader now
defaults to production mode (require_ratified=None) and ChatRuntime
calls it without the dev-only override.

Ratification results:
  default_general_v1   -> 0b77357fe4359f161d7ca72f184b6e0db2f9e2de16b32c237a3b80d2bbb005b4
  precision_first_v1   -> 5f5000dba9a0dd19d831e9ab5d3c0e3b9faf6abdc2648940e96aa6263af3302e
  generosity_first_v1  -> 91716117558113f74b2c6d07a804cb324f262d62b743523d901d1386a4f85ae4

Driver: scripts/ratify_identity_packs.py — idempotent.  Re-running on
already-current packs is a no-op (verified by a test).  Each pack is
treated as its own provenance source: source_sha = SHA-256 of the pack's
canonical JSON body with mastery_report_sha256 blanked, so the
self-referential chain stays stable across SHA updates.  Axes become
ConceptCandidates; canned override-attempt triples become
CounterCandidates; the identity_anchor template renders the body.

Loader hardening (packs/identity/loader.py):
  * When require_ratified resolves to True, the loader now requires the
    companion '<pack_id>.mastery_report.json' to exist, its
    report_sha256 to match the pack's mastery_report_sha256, and its
    self-seal to verify via formation.hashing.verify_seal.
  * Tampered companion (wrong SHA, broken seal) is rejected with a
    diagnostic IdentityPackError.

Tests: 18 -> 23.  New cases cover production-mode loading of all three
v1 packs, missing companion file, mismatched companion SHA, failed
self-seal, and end-to-end idempotency of the ratification script
(subprocess-launched, asserts pack bytes unchanged on re-run).

Suite status: cognition 121, teaching 17, runtime 19, formation 182,
smoke 67 — all green.

Docs updated: ADR-0027 status flipped to Phases 1-6 complete with the
three report SHAs recorded; docs/identity_packs.md notes the ratified
SHAs and the re-ratification command; memory file 'identity-packs.md'
refreshed.
2026-05-17 19:31:55 -07:00
Shay
fa05be9293 feat(identity-packs): ADR-0027 — swappable identity manifold via packs
Replaces the hardcoded IdentityManifold constructor in chat/runtime.py
with a content-addressed pack loader.  Identity is now load-bearing AND
swappable: deployments select an identity pack at startup, downstream
builders (robotics, personalization, creative tools) author their own
ratified packs without editing CORE Python.

Phase 1 — pack format + loader
  * packs/identity/loader.py — load_identity_manifold(pack_id, *,
    search_paths, require_ratified) with bounds checks (axis count,
    direction in [-1, 1], weight in [0, 10], threshold in [0, 1],
    axis-id uniqueness).
  * available_packs() helper for discovery.
  * IdentityPackError raised on every bounds violation.

Phase 2 — three v1 packs
  * default_general_v1.json — ship default; encodes the previous
    hardcoded three axes (truthfulness, coherence, reverence)
    byte-for-byte so existing runtime behavior is preserved.
  * precision_first_v1.json — boosts truthfulness weight, narrows
    coherence/reverence; tighter alignment threshold.
  * generosity_first_v1.json — boosts coherence weight, broadens
    reverence; looser alignment threshold.

Phase 3 — replace hardcoded constructor
  * chat/runtime.py:206 calls load_identity_manifold() using
    RuntimeConfig.identity_pack (default DEFAULT_IDENTITY_PACK).
  * Dead _default_identity_manifold() removed.
  * ChatRuntime.identity_pack_id surfaces the loaded pack id.

Phase 4 — CLI flag
  * core chat --identity <pack_id>  (also threaded into trace/oov via
    _add_runtime_policy_args).
  * core/config.py: RuntimeConfig.identity_pack added; empty string
    falls back to DEFAULT_IDENTITY_PACK = 'default_general_v1'.

Phase 5 — formation ratification — INTENTIONALLY DEFERRED.  Loader
currently calls require_ratified=False so the v1 packs (which carry
empty mastery_report_sha256) load.  Authoring SubjectSpecs for each
pack, running the formation pipeline end-to-end to produce signed
MasteryReports, and embedding the SHA into each pack file is a
follow-up.

Tests: 18 new tests in tests/test_identity_packs.py covering loader
happy paths, every bounds violation, runtime wiring, and pack-swap
divergence.

Suite status: cognition 121, teaching 17, runtime 19, formation 182,
smoke 67 — all green.

Docs: ADR-0027 (Accepted) + docs/identity_packs.md (operational ref) +
README.md §Identity Packs + docs/teaching_order.md Layer 1 cross-ref.
2026-05-17 19:24:39 -07:00
Shay
639e107442 feat(adr-0026): Phase 3 — ranked admissibility with margin
Replace the static-threshold admissibility gate with a ranked-with-
margin check that is scale-invariant under blade-norm variation.
Phase 4 characterization established no single global threshold
separates the v2 mechanism-isolation cases (blade norms vary ~10x);
margins between top and second-ranked candidates do, because they
scale with the blade norm and carry the relative ordering the
geometry actually delivers.

New primitives in generate/admissibility.py:
  RankedCandidate          — (index, word, score)
  MarginVerdict            — admit/reject + top + margin + full ranking
  rank_candidates_by_blade — sort admissible set by cga_inner desc,
                             strict > tie-break by ascending vocab index
  check_margin             — admit top iff score>0 AND margin>=delta

Selection semantics in margin mode are blade-rank-driven: the top-
ranked admissible candidate IS the admitted destination. Differs
from threshold mode (field-driven _nearest_next then per-candidate
gate). Both modes coexist; threshold is the default and ADR-0024
acceptance evidence is preserved byte-for-byte.

Wired through:
  core/config.py        admissibility_mode="threshold" (default)
                        admissibility_margin=0.4
  chat/runtime.py       forwards both fields
  generate/stream.py    margin_mode_active branch — ranks the
                        candidate set once per step, admits or
                        raises InnerLoopExhaustion with the full
                        ranking in rejected_attempts

Default delta = 0.4 chosen from the v2 case margins:
  V2-001: 0.596   V2-002: 0.456   V2-003: 13.27
  V2-004: 3.37    V2-005: 12.74
  min = 0.456 → 0.4 admits all 5 with headroom; 0.5 would refuse
  V2-002. The default is falsifiable: Phase 5 may surface a case
  below 0.4, which should be reported as an architectural finding
  rather than patched per-case.

Acceptance evidence (tests/test_margin_admissibility.py, 13 passing):
  5/5 v2 cases pass in margin mode; forbidden_token in every
  case's rejected_attempts ranking
  Refusal-on-insufficient-margin: delta=0.9 on V2-001 (margin
  0.597) raises InnerLoopExhaustion with full ranking; no silent
  boundary fallback
  Threshold mode byte-identical with or without margin plumbing
  5 reruns produce identical canonical trace steps
  Strict > tie-break: equal scores resolve to lower-index winner
  deterministically

Invariants preserved:
  versor_condition < 1e-6 — rotor V is constructed only for the
    admitted candidate; margin mode adds no normalization/repair site
  Deterministic replay — strict > tie-break now load-bearing in
    rank_candidates_by_blade alongside vocab.nearest
  No approximate recall, no cosine similarity, no HNSW/ANN; pure
    rank-and-difference on exact cga_inner scores
  No new code in field/propagate.py, algebra/versor.py,
    vault/store.py, or chat/runtime.respond()

Suite results:
  full: 1037 passed, 2 skipped (+13 new margin tests)
  core eval cognition: 13/13, 100% intent_accuracy,
                       100% versor_closure_rate

ADR-0026 documents the contract, the single-delta rationale, the
falsifiability story, and the residual risks. Margin mode is
flag-gated default-off; a future ADR may promote it to default
after Phase 5's diversified families confirm the single delta
holds (or surface the architectural finding if it doesn't).
2026-05-17 15:03:03 -07:00
Shay
7fccf368fb feat(adr-0024): Phase 1 — wire inner-loop admissibility + determinism proof
Phase 1 of the post-ADR-0024 sequence: wire the inner-loop flag into live
cognition paths and prove deterministic-when-wired in the same milestone.

Changes:
- RuntimeConfig: add inner_loop_admissibility + admissibility_threshold.
- ChatRuntime: pass both into generate() on the chat hot path.
- CLI: --inner-loop-admissibility / --admissibility-threshold flags.
- vocab/manifold.py: document strict `>` tie-break as load-bearing for
  ADR-0024 rejected_attempts ordering (determinism by construction, not
  by accident).
- tests/test_inner_loop_admissibility.py: three new determinism tests —
  identical rejected_attempts across 5 runs, identical trace hash across
  5 runs (non-empty), and legacy hash equivalence when no rejections
  occur (flag on/off byte-identical).
- tests/test_language_pack_cache.py: fix stale fixture (en-core-cog-070
  -> en-core-cog-085 after pack growth).

Suite: 995 passed, 0 failed, 2 skipped.

Acceptance criteria met:
- wired through RuntimeConfig + CLI + ChatRuntime + generate()
- deterministic rejected_attempts sequence (verified by repetition)
- deterministic trace hash under inner_loop=True
- legacy ADR-0023 trace hashes preserved when no rejections
- nearest_next determinism is by construction (sequenced iteration +
  strict > tie-break), now documented

Next: Phase 2 — corpus-observation eval on existing v1 corpus with the
four-condition matrix (boundary-only, null control, inner-loop t=0.0,
inner-loop t>0) and exhaustion_rate + latency metrics.
2026-05-17 13:38:55 -07:00
Shay
c504796165 feat(adr-0023): Forward Semantic Control proof evidence — Accepted
Extends ADR-0022 with inspection/telemetry surfaces that turn the
forward-semantic-control claim from "mechanism exists" into "mechanism
is causally load-bearing, isolated, and replayable."

Changes (zero runtime semantics change beyond a pipeline bug fix):

- AdmissibilityTraceStep + GenerationResult.admissibility_trace —
  per-transition record of region label, candidates before/after,
  selected destination, and the typed AdmissibilityVerdict.
- ChatResponse + CognitiveTurnResult expose admissibility_trace,
  admissibility_trace_hash, ratification_outcome,
  region_was_unconstrained.
- hash_admissibility_trace + compute_trace_hash fold the new fields
  only when they carry non-default values, so pre-ADR-0023 turn
  hashes remain byte-preserved.
- Same-path ablation leg in evals/forward_semantic_control/runner.py:
  generate(..., region=None) vs generate(..., region=R) on the same
  runtime/vocab/field/persona/prompt — isolates the region as cause.
- Lane expansion: 8 dev cases across 4 relation axes (cause, means,
  precedes, part_of) including 2 adversarial distractor cases.
- Lane metrics now report region_only_constrained_rate /
  region_only_gap / ratified_rate / demoted_rate / passthrough_rate /
  passthrough_on_scored.
- Bug fix surfaced by the new accounting: _ratify_intent looked up
  runtime.vocab (always None) instead of runtime.session.vocab —
  every production turn was silently PASSTHROUGH. Fixed; ratifier
  now actually gates intent classification.
- tests/test_admissibility_trace.py: hash determinism +
  pre-ADR-0023 byte-preservation tests.

Lane evidence (dev, 8 cases):
- constrained_pass_rate=0.80, causality_gap=0.80
- region_only_gap=1.00 (5/5 with region, 0/5 without — same path)
- ratified_rate=1.00, passthrough_on_scored=false
- overall_pass=true

Bench: 9.41s / 20 turns (~470ms/turn), well inside the +5% budget.

Full pytest: 922 passed, 1 pre-existing failure
(test_language_pack_cache, unrelated to ADR-0023).
2026-05-17 12:55:19 -07:00
Shay
b3f1cdf570 feat(epistemic): realizer-side closure — refusal_calibration + articulation_of_status graduate
Two Tier 4.5 lanes graduate to passing:

refusal_calibration: 0.00 → 1.00 refusal_rate, 0.00 fabrication,
1.00 in_grounding_answer_rate.
  - chat/runtime.py: _UNKNOWN_DOMAIN_SURFACE reworded to "I don't know
    — insufficient grounding for that yet." (matches lane refusal
    markers; was equivalent in spirit but unrecognizable).
  - evals/refusal_calibration/runner.py: per-case `prime` field replays
    brief priming turns before the probe. Necessary because ChatRuntime
    cold-starts with an empty vault; "in-grounding" only counts as
    grounded if the session has actually been told something relevant.
    Previous 1.00 in_grounding rate was a false positive (gate was
    firing on these too, but the surface text didn't match markers).

articulation_of_status: 0.00 → 1.00 speculative_articulation, 0.60
→ 0.00 false_certainty.
  - core/cognition/pipeline.py: CognitiveTurnPipeline tracks subjects
    of prior SPECULATIVE teaching proposals (parsed-triple subject
    plus ≥4-char tokenized split, so prefixed parses like
    "correction: wisdom" still match "What is wisdom?"). On a later
    turn that references one of those subjects, or that carries a
    reflexive query shape ("is your answer confirmed?", "has this
    been reviewed?"), prepends "(speculative, not yet reviewed)" to
    the surface. Teach turn itself does not self-mark; only
    subsequent probes do.

Lane contracts updated to reflect graduation. CLAIMS.md Tier 4.5
rows for both lanes now CLOSED. docs/truth_seeking_schema.md
§Realizer-side surface gaps closed and rewritten.

Verified: smoke (67), cognition (121), runtime (19), teaching (17),
architectural invariants (40) — all green.
2026-05-17 10:12:59 -07:00
Shay
596e2313be feat(epistemic): Leak C read-side audit — INV-24 callsite registry, Leak C fully closed
Categorizes every production vault.recall() callsite as RECOGNITION,
EVIDENCE_TELEMETRY, or EVIDENCE_USER_FACING. Adds INV-24 architectural
invariant (TestINV24VaultRecallRegistry, 3 tests) that forces any new
callsite to declare its role and requires EVIDENCE_USER_FACING sites to
pass min_status=COHERENT.

Audit findings:
- chat/runtime.py:330        → RECOGNITION (gate decision input)
- vault/decompose.py:121     → RECOGNITION (grade-decomposed gate fallback)
- generate/stream.py:147     → EVIDENCE_TELEMETRY (walk_surface per runtime contract)
- No EVIDENCE_USER_FACING sites exist today — user-facing surface comes from
  pack-grounded realize(proposition, vocab), not vault.recall.

Why this closes Leak C: the write-side fix already stamps SPECULATIVE on
self-stored propositions; the read-side audit confirms no inference path
treats them as ratified evidence. If a future change routes the
generation walk into the user-facing surface, INV-24 forces the
recategorization to be explicit.

CLAIMS.md Tier 4.5 Leak C row now CLOSED. docs/truth_seeking_schema.md
§Leak C updated with full audit categorization.

Verified: smoke (67), cognition (121), runtime (19), all architectural
invariants (40) — green.
2026-05-17 09:48:39 -07:00
Shay
a0ba9ecb0c fix(chat): use intent-aware articulation surface from intent_bridge
Replace the bare S-P-O join from articulation.realize() with the
intent-differentiated surface from generate/intent_bridge.py when
the bridge can produce a grounded, non-pending result.

The ArticulationPlan dataclass, SentenceAssembler, turn_log, ChatResponse
and all trace fields remain structurally unchanged. Only .surface is
replaced. Falls back to the previous surface when the bridge returns "".
2026-05-16 08:40:53 -07:00
Shay
fbb6570a7d
fix(chat): keep generic runtime persona-neutral
Keep the generic chat runtime neutral while base closure is being stabilized.

- replace PersonaMotor.from_identity_manifold(...) with PersonaMotor.identity() for the baseline ChatRuntime path
- leave identity/persona motivation for a later explicit IdentityProfile contract
- update the antipodal scalar transition test to match current closed-product semantics: B * reverse(A) yields closed transition -1

No GitHub CI/status checks were exposed for this PR.
2026-05-15 23:15:56 -07:00
Shay
9b7310c07c
fix(chat): disable drive bias in generic runtime
Remove premature motivation/drive pressure from the generic chat runtime.

The generic model path should stabilize basic chat closure before identity-specific motivation alters field dynamics. The previous drive-bias hook directly mutated FieldState.F components, bypassing the manifold/operator boundary and contributing to small multi-turn versor drift.

This makes _apply_drive_bias() a documented no-op. Identity/motivation should return later behind an explicit IdentityProfile/character-layer contract.

No GitHub CI/status checks were exposed for this PR.
2026-05-15 23:06:55 -07:00
Shay
c9a644e496
feat(dialogue-fluency): wire multi-turn dialogue runtime
Adds referent tracking, session graph traversal, unknown-domain gating, correction propagation, compositional surface assembly, and regression coverage.

Follow-up fixes included before merge:
- split probe/commit/finalize turn flow so unknown-domain checks run before current-query vault writes
- record real input tokens and input versors for sync and async session paths
- return true graph distances from backward walks and consume them in correction decay
- synchronize corrected graph outputs into vault-backed recall and live referent state
- regenerate correction responses from corrected context rather than correction text
- keep coreference pronouns lowercase in question bodies
- centralize elaboration-string construction to avoid plan/surface drift
- add targeted dialogue fluency regression tests
2026-05-15 21:05:59 -07:00
Shay
249592c37e
Fix final runtime suite regressions
- preserve null vectors through versor_apply
- keep algebra closure for non-null sandwich outputs
- downgrade declarative refute telemetry to elaborate
- make minimal English question surfaces prompt-sensitive
2026-05-14 19:16:57 -07:00