Commit graph

300 commits

Author SHA1 Message Date
Shay
a71b321a9a feat(adr-0055-0057): learning-loop demo — cold turn to grounded surface, end-to-end
`core demo learning-loop` (+ `--json`) walks a single prompt through the
full ADR-0055..0057 inter-session-memory architecture:

  S1. Cold turn          → universal disclosure, grounding_source=none
  S2. Discovery emission → DiscoveryCandidate to attached sink
  S3. Operator proposal  → real replay-equivalence gate, no regression
  S4. Operator accept    → TRANSIENT corpus only; active untouched
  S5. Same prompt        → teaching-grounded surface with the new chain

Before / after on the deterministic prompt "Why does thought exist?":

  before: [none]     I don't know — insufficient grounding for that yet.
  after:  [teaching] thought — teaching-grounded (cognition_chains_v1):
          cognition.thought; logos.internal. thought reveals meaning
          (cognition.meaning). No session evidence yet.

The active corpus on disk is byte-identical pre/post.  The demo writes
only to a transient corpus, then swaps `_CORPUS_PATH` for the after
turn — the same pattern the replay-equivalence gate uses.

- evals/learning_loop/run_demo.py — `run_demo(emit_json=False)` returns
  a structured `DemoReport` with both surfaces and per-scene detail.
- core/cli.py — `core demo learning-loop` target wired.
- tests/test_learning_loop_demo.py — 7 tests pin: full loop closes,
  before is ungrounded, after contains new chain atoms (thought /
  reveal / meaning), discovery emits ≥1, replay gate reports no
  regression, S4 byte-identical active + 1 line on transient, same
  prompt drives both surfaces.

Lane state: learning-loop-demo 7 new — green.  Demo runs in ~15s
end-to-end (cognition lane runs twice via replay gate).

No LLM provider has a published equivalent of this loop: per-fact
provenance from operator accept to surface, replay-equivalence gate
proving non-regression, byte-identical active state regardless of
outcome, full audit trail back to the originating cold turn.
2026-05-18 10:57:41 -07:00
Shay
6f4b2b7b2c feat(adr-0057): anti-regression demo — three-gate defense against learning harm
`core demo anti-regression` (+ `--json`) is a self-contained walkthrough of
the three independent gates that every reviewed-corpus extension must pass.
Designed for showcasing CORE's epistemic discipline to reviewers / industry
observers — no LLM provider has a published equivalent.

Scenes:
- S1. Eligibility predicate refuses an undetermined-polarity candidate
  before any replay is invoked.  ProposalError raised; no log row.
- S2. Replay-equivalence gate auto-rejects a regressing candidate with
  the named regressed metrics in the operator note.  Uses the documented
  `run_replay=` kwarg of `propose_from_candidate` to inject a controlled
  regression of the same `ReplayEvidence` shape the real gate produces.
- S3. Real `teaching.replay.run_replay_equivalence` runs the cognition
  public lane.  A replay-equivalent candidate reaches 'pending' — operator
  `--accept` is still required to write.

Each scene asserts the active corpus is byte-identical pre/post.

- evals/anti_regression/run_demo.py — `run_demo(emit_json=False)` returns
  a structured `DemoReport`; verbose human output by default, JSON on flag.
- core/cli.py — `core demo anti-regression` target wired alongside
  audit-tour / pack-measurements / long-context-comparison.
- tests/test_anti_regression_demo.py — 5 tests pin each scene's
  load-bearing claim + the corpus-byte-identical invariant.

Lane state: anti-regression-demo 5 new — green.  Demo runs in ~10s end-to-end.
2026-05-18 10:52:23 -07:00
Shay
3cad6686cc feat(adr-0057): operator supersession history view — closes the supersede loop
`core teaching supersessions` (+ `--json`) pairs each retired chain with its
active replacement.  Derived view over `audit_corpus()`; pure, read-only.

- teaching/audit.py — `SupersessionRecord` + `supersession_history(report)`
  returns retired→replacement pairs ordered by retired-line (disk order,
  oldest first).  Orphan supersessions (retired with no live entry carrying
  the matching `superseded_by` — e.g. chained retirements where the middle
  link itself was retired) surface as `replacement=None` so silent corpus
  drift is inspectable.
- core/cli.py — `core teaching supersessions [--json]`.  Exit 1 if any
  orphan is detected (catches silent drift in CI); 0 otherwise.
- tests/test_supersession_history.py — 7 tests pin empty-history,
  single-pair shape, chained-supersession surfaces both pairs, line-no
  ordering, orphan detection, JSON round-trip, no corpus mutation.

Lane state: smoke 67 / cognition 121 / supersession-history 7 new / supersede 13 /
audit 23 — green.  `core eval cognition`: unchanged (intent 100% / surface 100% /
term 91.7% / versor 100%).  Real corpus today reports `(no supersessions)`.
2026-05-18 10:40:38 -07:00
Shay
8d2c84a041 feat(adr-0057): operator supersede CLI — retire active chain by appended replacement
`core teaching supersede <old_chain_id> --subject ... --intent ... --connective ...
--object ... --review-date YYYY-MM-DD` is the second corpus mutation surface
(alongside accept_proposal). No replay gate — it's a deliberate operator action
that replaces a hand-authored or previously discovery-promoted chain.

- teaching/supersede.py — `supersede_chain()` orchestrator with pre-checks
  (review_date format, intent whitelist, pack-consistency via re-audit,
  no double-supersede, no self-supersede, no new-chain-id collision) and
  byte-identical rollback on post-audit failure.
- teaching/proposals.py — extended `append_chain_to_corpus` with optional
  `superseded_by` kwarg; remains the only function in the codebase that
  writes to the active teaching corpus.
- core/cli.py — `core teaching supersede` subcommand wired to the live
  `_CORPUS_PATH`; EPILOG updated with example.
- tests/test_supersede.py — 13 tests pin every gate, byte-identical
  rollback on rejection, append-only at disk level, audit-and-runtime
  parity after supersession, hand_authored provenance with
  `supersede(<old_chain_id>)` tag.

Lane state: smoke 67 / cognition 121 / teaching 17 / supersede 13 / audit 23 /
proposals 16 / contemplation 16 / contemplation-wiring 6 / discovery 24 — green.
`core eval cognition`: intent 100% / surface 100% / term 91.7% / versor 100% — unchanged.
2026-05-18 10:35:49 -07:00
Shay
e03ab4b609 feat(adr-0057): Phase C2 — TeachingChainProposal + replay gate + review CLI
The only path by which CORE extends its own active teaching corpus.
Closes ADR-0055 Phase C alongside ADR-0056's cognitive surface.

Three load-bearing calls (recorded in ADR-0057):
  1. Replay-equivalence is a precondition, not a permission;
     operator --accept remains required.
  2. Eligibility = polarity in {affirms, falsifies} AND at least
     one source='corpus' evidence pointer AND boundary_clean AND
     claim_domain != evaluative (unless --allow-evaluative) AND
     proposed_chain complete.
  3. Append-only proposal log; corpus history append-only too.

Changes
- teaching/proposals.py — TeachingChainProposal, ReplayEvidence,
  ProposalLog (event-sourced replay → current_state), eligibility
  predicate, propose_from_candidate, accept/reject/withdraw,
  append_chain_to_corpus (the sole corpus-write surface).  Uses
  TYPE_CHECKING guards to break the circular import with
  chat.pack_grounding.
- teaching/replay.py — run_replay_equivalence; swaps _corpus_index
  path to a tmp file, runs cognition lane on the active corpus
  AND a transient copy with the proposed chain appended, returns
  regressed-metrics list; trust-boundary assertion that the active
  corpus bytes are byte-identical pre/post.
- teaching/discovery.py — moved chat.pack_grounding /
  chat.teaching_grounding imports inside extract_discovery_candidates
  to break the cycle (was masked when chat.runtime was the entry
  point; surfaced by CLI entry).
- core/cli.py — three new subcommands:
    core teaching propose <candidate-jsonl-path> [--allow-evaluative]
    core teaching proposals [--state pending|accepted|rejected|withdrawn] [--json]
    core teaching review <proposal_id> --accept --review-date YYYY-MM-DD
    core teaching review <proposal_id> --reject [--note ...]
    core teaching review <proposal_id> --withdraw [--note ...]
- tests/test_teaching_proposals.py — 16 tests covering: every
  eligibility gate, proposal_id idempotency, append-only log,
  replay-equivalent stays pending, regression auto-rejects with
  named regressed metrics, --accept appends one line with typed
  Provenance, --accept refused on non-equivalent, state-machine
  blocks double-accept, real replay gate runs cognition lane
  twice and asserts byte-clean active corpus pre/post.

Invariants preserved
- versor_condition(F) < 1e-6 — C2 touches no algebra path.
- Active corpus bytes byte-identical regardless of replay outcome.
- No clock-time reads, no LLM, no async.
- Proposal-only — accept_proposal is the sole corpus-write path.

Lanes: smoke 67 / cognition 121 / runtime 19 / teaching 17 /
new proposals 16.  Cognition eval unchanged.

Open follow-ups (not in scope):
- supersession via operator review action
- cross-pack falsification arbitration (ADR-0056 Call 2 deferred)
- pack-data migration of frame-dependent connectives

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 10:23:14 -07:00
Shay
db6ce08589 feat(adr-0056): wire contemplation into live turn path (opt-in)
ChatRuntime.attach_contemplation(enabled=True) flips an opt-in
flag; when on, each emitted DiscoveryCandidate runs through
teaching.contemplation.contemplate before the sink writes the
JSONL line.  Default off ⇒ Phase B raw output preserved byte-
identical.

Trust boundary
- Contemplation is read-only over pack + corpus.
- Without an attached discovery sink the flag is inert (no hidden
  work — emission requires an observable destination).
- Active teaching corpus on disk byte-identical pre/post.

Lanes: smoke 67 / runtime 19 / cognition 121 / contemplation-
wiring 6 — all green.  Cognition eval unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 10:13:44 -07:00
Shay
f78def7f3a docs(adr-0056): mark Accepted in decisions index
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 10:10:53 -07:00
Shay
4e03a7f872 docs(adr-0056): Accepted (Phase C1 implemented at 4eecf73)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 10:07:27 -07:00
Shay
4eecf73a05 feat(adr-0056): Phase C1 — contemplation loop landed
Implements ADR-0056's cognitive surface: takes a Phase B
DiscoveryCandidate and returns an enriched candidate with composed
polarity, classified claim_domain, evidence pointers, and recursive
sub-questions.  No corpus mutation; no async; no LLM step.

Changes
- teaching/discovery.py: DiscoveryCandidate gains six C1 fields
  with defaults that preserve Phase B JSONL byte-equality.  Adds
  EvidencePointer, SubQuestion, ClaimDomain types.
- teaching/contemplation.py (new): contemplate(candidate) +
  canonical probe order (vault → pack → corpus), deterministic
  decomposition over corpus-known intent objects, composition
  rules from ADR-0056 §Composition, bounded-depth failsafe with
  recursion_overflow audit signal.  Vault probe is injectable;
  None means no vault contribution this pass.
- tests/test_contemplation.py (16 tests): determinism (byte-
  identical JSONL), no input/corpus mutation, empty pack+corpus
  termination with gap-recorded sub-question, factual affirming
  composition, direct same-pack contradiction → falsifies, mixed
  evidence → undetermined + domain upgrade, recursion overflow,
  frame-dependent connective → relational, Phase B byte-equality
  preserved on uncontemplated candidates, sub_id stability,
  evidence pointer admissibility, vault probe injection +
  exception isolation.

Invariants preserved
- versor_condition(F) < 1e-6 — C1 touches no algebra path.
- No corpus / pack / runtime mutation — trust boundary intact.
- No clock-time, no LLM, no stochastic sampling, no async.

Lanes
- smoke 67, cognition 121, runtime 19, teaching 17, contemplation 16.
- core eval cognition: intent 100% / surface 100% /
  term_capture 91.7% / versor 100% — unchanged.

Open questions stay open: frame-dependent connective table
authorship (v1 lives as a small constant in contemplation.py
pending pack-data migration), person-axis intent classification
for auto-evaluative, recursion-overflow telemetry shape, sub-
question deduplication.  None block C1.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 10:06:18 -07:00
Shay
f1121b5822 docs(adr-0056): Contemplation loop C1 — Proposed
Splits ADR-0055 Phase C into:
- C1 (this ADR): cognitive contemplation loop — question
  decomposition + polarity (affirms/falsifies/undetermined) +
  claim_domain typing (factual/relational/evaluative)
- C2 (future ADR): review-and-apply — TeachingChainProposal,
  replay-equivalence gate, corpus append-on-accept

Documents four load-bearing design calls with explicit reasoning
so future sessions can re-derive without re-arguing:

1. Stopping condition: record-the-gap-and-stop primary, bounded
   depth failsafe; failsafe firing emits recursion_overflow audit
   signal — never silent truncation.
2. Falsification evidence: reviewed-only, same pack family;
   session-tier contests but does not falsify. Cross-pack
   arbitration deferred.
3. Order: C1 before C2. Reversed instinct to land 'small thing
   first' — C2 alone is useless without enriched input; C1
   physically cannot mutate corpus until C2 wires the apply path.
4. Sync, not async. CORE hot path is deterministic; concurrency
   overhead exceeds probe cost on local-only probes. Async
   deferred to a future ADR if a blocking probe surface emerges.

Trust boundary: C1 never mutates the corpus. C1 reads pack,
corpus, vault, and most recent TurnEvent; writes only to the
existing Phase B discovery sink. Gap-recorded sub-questions
emit as new top-level candidates on the same sink — recursion
reified into the stream.

Maps directly onto user-stated framing recorded verbatim in the
ADR:
- 'contemplation always starts with a question' → candidate is
  the posing; contemplate() is the answering
- 'truths and/or falsities' → polarity on the chain itself
- 'remain humble' → claim_domain with escalating evidence
  thresholds, mandatory hedge for evaluative
2026-05-18 08:52:43 -07:00
Shay
07d35c0f54 feat(adr-0055): Phase B — DiscoveryCandidate emission from turn loop
Lands the first deterministic trigger of the discovery → reviewed-
memory loop. Candidates are structured evidence; emission is
opt-in via attach_discovery_sink and NEVER mutates the active
teaching corpus.

- teaching/discovery.py: DiscoveryCandidate dataclass + pure
  extract_discovery_candidates(turn_event, intent, subject) rule
  firing. Phase B fires only the would_have_grounded trigger:
    grounding_source == "none"
    AND intent ∈ {CAUSE, VERIFICATION}
    AND subject lemma in ratified cognition pack
    AND (subject, intent) NOT in active corpus
  candidate_id = SHA-256 of canonical JSON payload — replay-stable.
  Other DiscoveryTrigger literals (successful_comparison,
  hedge_acknowledged, oov_resolved_via_decomp) are reserved for
  later phases.

- teaching/discovery_sink.py: DiscoveryCandidateSink protocol,
  DiscoveryBufferSink (in-memory), DiscoveryMonthlyFileSink
  (append-only JSONL, <root>/<YYYY>/<YYYY-MM>.jsonl rollover,
  injectable clock).

- chat/runtime.py: opt-in attach_discovery_sink, post-turn
  emission inside _stub_response only when caller threads
  classified intent forward (gate-fire fall-through site).
  Intent classification at the call site reuses the same
  deterministic classifier already invoked by
  _maybe_pack_grounded_surface for the empty-vault English path.

Trust boundary: candidates write to a separate sink/file path
only; the active corpus on disk is never touched. Tests
explicitly assert corpus bytes are byte-identical before and
after a candidate-emitting turn.

Tests: tests/test_discovery_candidates.py — 24 tests covering
pure-predicate rule firing, every short-circuit path,
deterministic candidate_id, sink opt-in, runtime parity with no
sink, monthly rollover semantics, append-only behaviour, no
corpus mutation.

Lanes: smoke 67, cognition 121, runtime 19, teaching 17, packs 6
— all green. Cognition eval metrics unchanged on dev / public /
holdout splits. versor_condition < 1e-6 invariant untouched.
2026-05-18 08:26:04 -07:00
Shay
7aa77806f9 feat(adr-0055): Phase A — teaching corpus audit, supersession, typed provenance
Lands the three load-bearing pieces of ADR-0055 Phase A so later
phases (DiscoveryCandidate, TeachingChainProposal) have a safe
substrate to write into.

- teaching/audit.py: pure, deterministic re-parse of the reviewed
  corpus with same gates as the runtime loader but keeps drop
  reasons (invalid_json, missing_required_field:*, unsupported_intent,
  pack_missing_subject, pack_missing_object, superseded_by:*).
- teaching/provenance.py: typed Provenance(adr_id, source,
  review_date, raw); legacy "reviewed" maps to "hand_authored" so
  current corpus reports the canonical enum without a file rewrite.
- chat/teaching_grounding._corpus_index honors superseded_by —
  active view drops superseded entries while disk preserves history.
- core teaching audit CLI subcommand (--json optional); exits 1 on
  any drop so CI catches silent corpus shrinkage from pack swaps.

Observable behaviour unchanged: corpus is 10/10 loaded, all five
core lanes green (smoke 67, cognition 121, runtime 19, teaching 17,
packs 6), cognition eval metrics identical on dev / public /
holdout splits. versor_condition < 1e-6 invariant untouched.

Tests: tests/test_teaching_audit.py — 23 tests covering provenance
parser, real-corpus determinism, every drop-reason path,
supersession semantics, runtime/audit parity, read-only contract.
2026-05-18 08:15:23 -07:00
Shay
0f797e2940 docs(adr-0055): inter-session memory — reviewed discovery promotion (Proposed)
Phased design for closing the inter-session learning loop without a
parallel learning path:

- Phase A: make today's 4-tier story load-bearing (audit CLI,
  active-set view via superseded_by, typed provenance enum)
- Phase B: DiscoveryCandidate emission from the turn loop —
  deterministic rule-firing on the audit trail, never writes the
  corpus
- Phase C: TeachingChainProposal — sibling to PackMutationProposal,
  proposal-only, replay-equivalence gate on dev+public
- Phase D: epistemic-tier guard (only COHERENT evidence promotes)
- Phase E: curriculum integration via formation review

Non-goals named explicitly: no embeddings, no DB storage, no
automatic identity/safety/ethics mutation, no opaque LLM step, no
removal of human reviewer.

Status Proposed; later ADRs land each phase against the verification
contracts named here.
2026-05-18 08:09:40 -07:00
Shay
6b25069da8 feat(adr-0054): vault recall indexing/batching + holdout split wired
Two doctrine-aligned CLAUDE.md items closed together.

Part 1 — vault indexing + batching (item #4):
- VaultStore lazy _matrix_cache (invalidated on store / reproject /
  eviction); vault_recall(prebuilt_matrix=...) skips deque→ndarray
  rebuild on hot path
- New vault_recall_batch + VaultStore.recall_batch — B queries
  scored in one component-serial sweep, bit-identical to per-query
  vault_recall (3 seeds × 7 queries × N=137 parity test)
- No approximation, no hot-path repair, scoring arithmetic
  unchanged

Part 2 — holdout split wired:
- LaneInfo.holdout_cases_path resolves plaintext holdouts in fixed
  priority; sealed (.age) holdouts stay in holdout_runner
- framework.run_lane(split="holdout") + argparse --split choices
- First official cognition holdout numbers: 19 cases, intent 100%,
  surface 94.7%, term_capture 70.8%, versor 100% — single miss is
  predicted correction_truth_040 (ADR-0053 scope-limit)

Tests: 21 new vault tests + 10 new framework tests. Lanes: smoke
67, cognition 121, runtime 19, teaching 17, packs 6, algebra 132 —
all green. versor_condition < 1e-6 invariant preserved.
2026-05-18 07:58:57 -07:00
Shay
e975faf8a8 feat(adr-0053): cognition lane closure — corpus expansion + CORRECTION acknowledgement
Closes both cognition splits at 100% surface_groundedness.  Three
parts:

1. Teaching corpus expansion (no code).  cognition_chains_v1.jsonl
   grows 3→10 chains.  3 close dev-split misses (correction,
   creation, light-as-VERIFICATION); 4 pre-empt the analogous
   holdout pattern (CAUSE/VERIFICATION on truth + wisdom).  Every
   subject/object is a pack lemma; every connective is a recognised
   humanize_predicate predicate.

2. CORRECTION acknowledgement branch.  New
   `pack_grounded_correction_surface()` in chat/pack_grounding.py,
   wired into `_maybe_pack_grounded_surface` for cold-start
   CORRECTION intents.  Fixed-template surface with distinct
   trailing disclosure ("No prior turn in this session to correct
   yet.") — distinguishes the cold-start acknowledgement from the
   DEFINITION-of-correction surface.  The post-correction reviewed-
   teaching path in teaching/correction.py is unchanged.

3. Diagnostic memory.  Saves the dev-split generalization finding:
   the ADR-0048→0052 chain is NOT overfit.  Public/dev gap was
   teaching-corpus content coverage, not architecture.

Eval deltas (both splits run, post-ADR-0053):
                       public   dev
  intent_accuracy        100%   100%   (=)
  surface_groundedness   100%   100%   SATURATED
  term_capture_rate    91.7%  78.6%
  versor_closure_rate    100%   100%   (=)

Public surface_groundedness: 92.3% → 100%   (+7.7 pp)
Dev    surface_groundedness: 69.2% → 100%   (+30.8 pp)

Tests: tests/test_pack_grounded_correction.py (15 new tests).
Lanes green: smoke (67), cognition (121), runtime (19),
teaching (17), packs (6).

Scope limits: holdouts (19 cases) not yet in the official
`core eval cognition` runner (--split accepts only {dev, public});
the CORRECTION surface does not yet echo the corrected-subject
lemma (relevant only for holdout case `correction_truth_040`).
2026-05-18 07:43:39 -07:00
Shay
822d8e1672 docs(adr): index ADR-0051 + ADR-0052 in decisions README
Add the two rows the orchestrator deferred while the parallel
subagent worktrees were in flight.  Both ADRs were merged in
preceding commits; this lands the README index entries that
were intentionally fenced out of each subagent's scope to
avoid merge-conflict noise.
2026-05-18 07:30:14 -07:00
Shay
0d854ff387 merge: ADR-0052 teaching-grounded CAUSE/VERIFICATION surface 2026-05-18 07:28:12 -07:00
Shay
e6a9662b5b merge: ADR-0051 trust-boundary hardening pass 2026-05-18 07:28:05 -07:00
Shay
c6ade6c76f feat(adr-0052): teaching-grounded CAUSE/VERIFICATION surface 2026-05-18 07:13:43 -07:00
Shay
140b6fea37 feat(adr-0051): trust-boundary hardening pass 2026-05-18 07:09:55 -07:00
Shay
ecd580479a feat(adr-0050): pack-grounded COMPARISON surface
Sibling to ADR-0048's DEFINITION/RECALL pack-grounded surface for
the COMPARISON intent.  `pack_grounded_comparison_surface(a, b)` in
`chat/pack_grounding.py` composes a deterministic side-by-side
surface from both lemmas' pack `semantic_domains`, joined by the
fixed connective "contrasts with":

  "{a} (d_a1; d_a2) contrasts with {b} (d_b1; d_b2) — pack-grounded
   ({pack_id}). No session evidence yet."

`chat/runtime.py:_maybe_pack_grounded_surface` gains a COMPARISON
branch that runs before the DEFINITION/RECALL check.  Engages only
when both `intent.subject` and `intent.secondary_subject` are pack
lemmas and differ (identical-lemma comparison defers to disclosure).
Order-sensitive by design — matches the graph-layer's directional
CONTRAST edge.

Cognition eval (13-case public split):
  surface_groundedness  61.5% → 69.2%  (+7.7 pp)
  term_capture_rate     50.0% → 58.3%  (+8.3 pp)
  intent_accuracy            100.0%        (=)
  versor_closure_rate        100.0%        (=)

Case lifted: comparison_memory_recall_030 ("Compare memory and
recall").  Remaining unlift cases (CAUSE×2, VERIFICATION×1,
CORRECTION×1) need teaching-store chains or operator-driven
inference — pack lookup cannot supply causal explanations,
verifications, or corrections without fabrication.

Tests: tests/test_pack_grounded_comparison.py (15 tests).
Lanes green: smoke (67), cognition (121), runtime (19), algebra
(132), teaching (17), packs (6).
2026-05-18 06:59:53 -07:00
Shay
c8037cfa0d feat(adr-0049): head-noun subject extraction in intent classifier
Add a deterministic, pack-agnostic post-processor in `generate/intent.py`
that runs after the `_RULES` table fires:

- DEFINITION / RECALL / PROCEDURE: strip trailing punctuation + leading
  articles; preserve multi-word noun phrases
- CAUSE / VERIFICATION: additionally strip leading aux verbs; return
  the head noun

Closed-set frozen sets (`_ARTICLES`, `_AUX_VERBS`) make the transform
inspectable. No pack load, no algebra change — touches only
`DialogueIntent.subject`.

Cognition eval (13-case public split):
  surface_groundedness  46.2% → 61.5%  (+15.3 pp)
  term_capture_rate     33.3% → 50.0%  (+16.7 pp)
  intent_accuracy            100.0%        (=)
  versor_closure_rate        100.0%        (=)

Two cases lift through the ADR-0048 pack path
(definition_procedure_023, definition_relation_026 — both
"What is a X?" → subject=X via article stripping). CAUSE / VERIFICATION
subjects are now clean head nouns, foundational for future COMPARISON
pack path / teaching-store inference.

Tests: tests/test_intent_subject_extraction.py (30 tests).
Lanes green: smoke (67), cognition (121), runtime (19), algebra (132),
teaching (17), packs (6).
2026-05-18 06:51:46 -07:00
Shay
98a045337d Merge ADR-0048: pack-grounded surface for cold-start DEFINITION/RECALL
Cognition eval delta: surface_groundedness 15.4%→46.2%,
term_capture_rate 0.0%→33.3%, intent and closure unchanged.

Lanes green: smoke 67 / cognition 121 / runtime 19 / algebra 132 /
teaching 17 / packs 6.
2026-05-18 06:36:23 -07:00
Shay
c28e107dc7 feat(adr-0048): pack-grounded surface for cold-start DEFINITION/RECALL
Closes the surface-grounding gap isolated by ADR-0047's
characterisation.  Adds the ratified cognition pack as a second
grounding source alongside the session vault.

== chat/pack_grounding.py (new) ==

Loads en_core_cognition_v1's lexicon once (cached; immutable pack)
and exposes:

  pack_grounded_surface(lemma) -> str | None

Returns a deterministic, fully pack-sourced surface:

  "{lemma} — pack-grounded ({pack_id}): {d1}; {d2}; {d3}. No session evidence yet."

Every visible atom is the lemma or a verbatim semantic_domains
string from the pack.  No rewording, no synthesis, no LLM.

== chat/runtime.py ==

_stub_response gains optional pack_grounded_surface= parameter.
_maybe_pack_grounded_surface routes to the pack only when all four
hold: gate_source=="empty_vault", output_language=="en",
intent.tag in {DEFINITION, RECALL}, and intent.subject is a pack
lemma.  Safety/ethics refusal still takes priority above this branch.

ChatResponse and TurnEvent gain grounding_source ∈ {vault,pack,none}.
Main walk path tags responses "vault".

== core/cognition/pipeline.py ==

gate_fired detection moved from string equality on the universal
disclosure to provenance:

  gate_fired = response.vault_hits == 0 and response.grounding_source != "vault"

Same intent (suppress realizer template on gate-fired turns),
broader stub-path surface set.

== Characterisation (core eval cognition, 13-case public split) ==

  Metric                  Pre        Post     Δ
  intent_accuracy        100.0%     100.0%    0
  surface_groundedness    15.4%      46.2%   +30.8 pp
  term_capture_rate        0.0%      33.3%   +33.3 pp
  versor_closure_rate    100.0%     100.0%    0

Lift is non-uniform by design: only single-lemma DEFINITION/RECALL
on pack-known English subjects engage.  CAUSE/COMPARISON/VERIFICATION
and multi-word OOV subjects still return the universal disclosure —
fabricating those would violate the no-LLM-fallback doctrine.

== Tests ==

  tests/test_pack_grounding.py                          18 passed
  tests/test_semantic_realizer_integration.py (updated) 1 stub-path test
    pinned to the broader contract: surface is either universal
    disclosure or pack-grounded; never the realizer template.

== Lanes ==

  smoke 67  cognition 121  runtime 19  algebra 132
  teaching 17  packs 6

versor_condition(F) < 1e-6 invariant unaffected (no algebra changes).
2026-05-18 06:36:10 -07:00
Shay
5194a14788 Merge ADR-0047: wire forward graph constraint into the chat hot path
Closes ADR-0046's deferred follow-up.  Opt-in via
RuntimeConfig.forward_graph_constraint (default False).

A/B characterisation on the 13-case cognition lane shows wiring is
correct and safe, six cases produce a non-trivial constraint, and
the lane's surface_groundedness / term_capture_rate gap is downstream
of propagation — isolating the next ADR's scope to realizer / surface
assembly.

Lanes green: smoke 67 / cognition 121 / runtime 19 / algebra 132 /
teaching 17 / packs 6.  core eval cognition unchanged with the flag
off (default), unchanged with the flag on (constraint engages but
does not alter surface for this lane).
2026-05-18 06:18:30 -07:00
Shay
f47a85a3e7 feat(adr-0047): wire forward graph constraint into the chat hot path
Closes ADR-0046's deferred follow-up: convert the PropositionGraph
into an AdmissibilityRegion BEFORE generate() runs on the live
chat path.

== generate/intent_bridge.py ==

New public helper:

    build_graph_from_input(text, plan) -> PropositionGraph

Same internal call as _build_graph_from_intent, without the
post-generation ground_graph step — suitable for forward use.

== chat/runtime.py ==

When the new flag is on and output language is English, build the
graph and the region before generate() and pass it via region=.
Empty / fully OOV graphs return AdmissibilityRegion(allowed_indices=None),
which generate() treats as unconstrained — the change is a true
no-op when the graph carries no in-vocab anchors.

== core/config.py ==

RuntimeConfig.forward_graph_constraint: bool = False

Default False preserves all pre-ADR-0046 behaviour and the ADR-0024
honest-refusal contract.  A first attempt wired the constraint
unconditionally; 15 tests failed with InnerLoopExhaustion because the
intent-derived graph's CGA neighbourhood doesn't intersect the walk's
candidate pool with top_k=8 on the current packs.  The honest answer
is not to widen top_k until the failure goes away nor to silently
relax — both erase the architectural information that the geometry
of the graph and the geometry of the walk are not yet co-located.
Opt-in preserves ADR-0024 and follows the ADR-0022→0026 transition-
window pattern.

== Characterisation (core eval cognition, 13-case public split) ==

A/B with the flag toggled:

  Metric                  OFF      ON      Δ
  intent_accuracy        100.0%   100.0%   0
  surface_groundedness    15.4%    15.4%   0
  term_capture_rate        0.0%     0.0%   0
  versor_closure_rate    100.0%   100.0%   0
  InnerLoopExhaustion       0        0     0
  non-trivial constraint   n/a    6 / 13   —

Findings:
- Wiring is correct and safe (no exhaustions, closure unchanged).
- Single-token in-vocab subjects engage the constraint
  (light/knowledge/meaning/memory/correction).
- Multi-word OOV subject phrases produced by the intent classifier
  fall through to unconstrained — this is the existing intent-
  classifier contract surfacing into geometry, not a constraint bug.
- Restricting which tokens the walk may visit did not change
  surface_groundedness or term_capture_rate on this lane.  The
  surface-grounding gap therefore lives downstream of propagation
  — in the realizer / surface-assembly / dialogue-role path — and is
  the next load-bearing pull.  This isolates the next ADR's scope.

== tests/test_forward_graph_constraint_wiring.py (5 tests) ==

  - DEFAULT_CONFIG.forward_graph_constraint is False
  - Default runtime answers without InnerLoopExhaustion
  - Opt-in runtime answers on a short benign input
  - Graph builder + build_graph_constraint produce a labelled
    AdmissibilityRegion ("graph:unconstrained" or "graph:<root_id>")
  - Flag is observable on the frozen RuntimeConfig

== docs/decisions/ ==

  - ADR-0047 ratifies the wire-up, opt-in rationale, and A/B numbers.
  - README index updated; the Pillar 1→2→3 section now reflects both
    the primitive (ADR-0046) and the live wiring (ADR-0047), and
    names the next pull (realizer / surface assembly) explicitly.

Verification (this branch):

  tests/test_forward_graph_constraint_wiring.py    5 passed
  tests/test_graph_constraint.py                   8 passed
  core test --suite smoke                         67 passed
  core test --suite cognition                    121 passed
  core test --suite runtime                       19 passed
  core test --suite algebra                      132 passed
  core test --suite teaching                      17 passed
  core test --suite packs                          6 passed
  core eval cognition                            metrics unchanged from main

versor_condition(F) < 1e-6 invariant unaffected.
2026-05-18 06:18:10 -07:00
Shay
3067e7ddb2 Merge ADR-0046: PropositionGraph as forward AdmissibilityRegion
Brings in the original ADR-0046 work (Perplexity-coauthored) plus the
mergeability fix.  The merge preserves both commits so the history
records what was tried, what broke, and what fixed it.

Lanes green on the merged tree:
  smoke 67  cognition 121  runtime 19  algebra 132  teaching 17  packs 6
  core eval cognition  intent_accuracy=100%  versor_closure_rate=100%
2026-05-18 05:58:12 -07:00
Shay
c01ad748c8 fix(adr-0046): make forward-graph-constraint branch mergeable
The original adr-0046 commit was never run.  Fixes:

- generate/graph_constraint.py: import RegionSource (was the
  non-existent AdmissibilitySource).
- tests/test_graph_constraint.py + demo_01: load pack
  "en_core_cognition_v1" (was "en", which is not a pack ID).
- demo_03: read JsonlBufferSink.lines as a list attribute, not a
  method call.
- demo_04 (exact_recall_scale): DROPPED.  The construction used
  raw standard_normal vectors through unitize_versor and asserted
  cga_inner self-similarity is the population max.  Cl(4,1) has
  mixed signature — cga_inner is not self-maximising for arbitrary
  unitized random vectors — and the demo failed at N=10 000 in
  exactly the way the construction predicts.  The exact-recall
  claim's correct home is ADR-0045 (real vault path, properly
  constructed versors, N up to 100k = 100%).

Doc/index updates:

- ADR-0046 trimmed to three demos, with an explicit note on the
  dropped demo's geometric error and the cross-reference to
  ADR-0045.
- ADR-0046 verification block updated with measured lane numbers
  (smoke 67 / cognition 121 / runtime 19 / algebra 132 /
  teaching 17 / packs 6; core eval cognition unchanged).
- ADR-0046 cross-references ADR-0018 (intent_bridge source of the
  graph) and ADR-0022→ADR-0026 (AdmissibilityRegion contract).
- docs/decisions/README.md: ADR-0046 added to the index and to a
  new "Pillar 1 → 2 → 3 coupling" section linking the graph
  constraint to the existing forward-semantic-control chain.
- evals/industry_demos/__init__.py: invocation list trimmed to
  the three real entry points; removed the aspirational
  "core demo …" subcommands that were never wired.

Verification on this branch:
  tests/test_graph_constraint.py        8 passed
  evals/industry_demos/demo_01..03      exit 0 each
  core test --suite smoke              67 passed
  core test --suite cognition         121 passed
  core test --suite runtime            19 passed
  core test --suite algebra           132 passed
  core test --suite teaching           17 passed
  core test --suite packs               6 passed
  core eval cognition                 intent 100%, versor_closure 100%
2026-05-18 05:57:46 -07:00
Shay
83443bd071 feat(adr-0046): PropositionGraph as forward constraint + industry demos
Closes the structural gap identified in the 2026-05-17 assessment:
the PropositionGraph was a post-hoc descriptor of what the field walk
already produced.  It is now a forward constraint that shapes what the
walk is ALLOWED to produce.

== generate/graph_constraint.py (new) ==

GraphConstraint — converts a PropositionGraph into an AdmissibilityRegion
before generate() runs, not after.  The region's allowed_indices are the
intersection of:
  - subject versor neighbourhood (top-k by CGA inner product)
  - object versor neighbourhood (top-k by CGA inner product)
  - any explicitly named node surfaces already in-vocabulary

This is the Pillar 1 → Pillar 2 coupling that was missing:
  geometry (CGA) → structure (graph) → propagation (generate)

build_graph_constraint(graph, vocab, *, top_k) is the public entry.
The region label encodes the graph's root node IDs so the admissibility
trace identifies the constraint source.

== generate/stream.py (updated) ==

generate() already accepts an AdmissibilityRegion.  No new API needed —
graph_constraint.build_graph_constraint() produces one.

== evals/industry_demos/ (new) ==

Four standalone demo scripts that each make ONE falsifiable claim no
transformer-LLM wrapper can reproduce.  Each script runs independently
via `python -m evals.industry_demos.<name>` and exits 0 on pass / 1 on
fail.  Each prints structured evidence to stdout.

  demo_01_forward_constraint.py
    Claim: When the PropositionGraph names subject=light, obj=truth, the
    generation walk is constrained to the CGA neighbourhood of those
    versors BEFORE any tokens are produced.  The allowed_indices set is
    computed from geometry, not from a prompt filter.  Demonstrated by
    showing the AdmissibilityRegion is non-trivial (< full vocab) and
    that all generated tokens score positive CGA inner product against
    the constraint field.

  demo_02_geometry_drives_identity.py
    Claim: Swapping the identity pack (precision_first vs generosity_first)
    on identical input produces structurally different surfaces via the
    manifold alignment path — not via a system-prompt swap.  Demonstrated
    by running two ChatRuntime instances with different identity_pack IDs
    on the same text, showing hedge_rate and identity_score.alignment
    differ, and that the manifold alignment_threshold differs at the
    algebra level (not just the text level).

  demo_03_deterministic_audit.py
    Claim: Three independently constructed ChatRuntime instances on the
    same input produce byte-identical JSONL audit lines.  Demonstrated
    by attaching JsonlBufferSink to each, running chat(), and asserting
    hash equality of the emitted lines (modulo the 'turn' field which is
    per-instance sequential).  This is architectural determinism — not
    seeded randomness.

  demo_04_exact_recall_scale.py
    Claim: CGA vault recall is exact (100%) at N=100, N=1_000, N=10_000.
    The needle versor is recovered at rank-1 by cga_inner scan regardless
    of vault size.  No approximate nearest-neighbour index.  No FAISS.
    No degradation curve.  Demonstrated inline with timing so the
    linear-scan cost is visible alongside the 100% recall.

== tests/test_graph_constraint.py (new) ==

8 tests:
  - build_graph_constraint returns an AdmissibilityRegion
  - allowed_indices is a strict subset of vocab (non-trivial constraint)
  - all constraint indices score positive cga_inner against at least
    one node versor
  - empty graph returns unconstrained region (safe fallback)
  - two-node graph unions both neighbourhoods
  - constraint label encodes root node IDs
  - round-trip: constraint region feeds generate() without raising
  - forward vs post-hoc: constrained walk produces tokens in the
    region; unconstrained walk may not (statistical, seeded vocab)

Co-Authored-By: Perplexity AI
2026-05-17 23:58:30 -07:00
Shay
283680f110 feat(adr-0044, adr-0045): domain ethics pack + long-context comparison
ADR-0044 — Medical / clinical ethics pack (worked-example domain pack).
Ships packs/ethics/medical_clinical_ethics_v1.json with six commitments
partitioned across all three remediation tiers:
  - refuse: no_dosing_recommendation, no_emergency_triage_authority
  - hedge:  defer_diagnosis_to_clinician, surface_evidence_grade
  - audit:  disclose_no_clinician_relationship, respect_patient_autonomy

Ratified end-to-end through scripts/ratify_ethics_pack.py (PACK_IDS
extended).  Production-mode load via load_ethics_pack succeeds.
ChatRuntime composition includes universal safety floor + every medical
commitment.  tests/test_medical_clinical_ethics_pack.py (8 tests) gates
file existence, sealed report, disjoint refusal/hedge lists, and
pack-swap visibility (default pack does NOT carry medical commitments).

ADR-0045 — Long-context recall: CORE vs transformer baselines.
Adds evals/long_context_cost/comparison_runner.py with a deterministic
needle-in-a-haystack measurement at N ∈ {100, 1_000, 10_000, 100_000}.
CORE recall = 100% at every tested N by exact cga_inner scan.

Paired with frozen citations of published transformer NIAH numbers in
evals/long_context_cost/baselines/transformer_long_context.json:
Claude 2.1 (200k, 50%), GPT-4 Turbo 128k (~71%), Gemini 1.5 Pro (99.7%),
NVIDIA RULER (varies).  Each citation carries source + url.

The two components measure different inputs (synthetic versors vs NL
needles) and are not directly comparable benchmark-for-benchmark.  The
comparison is at the architectural level — exact-scan recall vs
attention-based probabilistic recall.  Scope and limits documented in
the ADR.  tests/test_long_context_comparison.py (5 tests) gates schema,
CORE recall == 100%, and baseline citation presence.

CLI integration: two new demo targets with study-grade preambles.
  - core demo pack-measurements          (ADR-0043 — wired)
  - core demo long-context-comparison    (ADR-0045)
README + docs/PROGRESS.md cheatsheets updated.  docs/decisions/README.md
index extended with ADR-0044 + ADR-0045; pack-layer chain title now
"ADR-0027 through ADR-0045".

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 22:31:47 -07:00
Shay
4ba1ef2da3 feat(adr-0043): Phase-2 pack measurements — claims → numbers
Converts the load-bearing claims of the ADR-0027→0042 pack-layer chain
into CI-enforced numbers across the three ratified identity packs
(default_general_v1, precision_first_v1, generosity_first_v1).

Two new pack-driven runners + an orchestrator:

- evals/identity_divergence/pack_runner.py — drives real
  SentenceAssembler + SurfaceContext (no mocks) across all three
  packs over 10 cases × 5 alignment bands; publishes per-pack
  bare/hedge/qualifier rates and pairwise distinct_rate.

- evals/refusal_calibration/pack_runner.py — runs the existing
  grounding-refusal lane via RuntimeConfig(identity_pack=...);
  publishes per-pack refusal_rate/fabrication_rate and a
  pack_invariant_gate flag asserting byte-identical cold-start
  surfaces across packs.

- scripts/publish_pack_measurements.py — combined publisher
  emitting evals/results/phase2_pack_measurements.json.

Baseline numbers (2026-05-17):
- precision_first hedge_rate=0.60, qualifier_rate=0.20
- generosity_first hedge_rate=0.20, qualifier_rate=0.00
- default_general hedge_rate=0.40, qualifier_rate=0.00
- pairwise distinct_rate ∈ [0.40, 0.80]
- refusal_rate=1.00, fabrication_rate=0.00 for all three packs
- pack_invariant_gate=True

6 tests in tests/test_pack_measurements_phase2.py lock the schema +
load-bearing flags + the structural inequality
precision.hedge_rate > generosity.hedge_rate. If identity packs
get wired into the cognition gate, pack_invariant_gate flips and
the suite fails.

ADR-0043 documents the numbers, the extended marker rationale, and
the trade-offs. README index updated with ADR-0043 row and chain
title bumped to "ADR-0027 through ADR-0043".

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 22:19:24 -07:00
Shay
294cfc3576 feat(adr-0042): audit-tour demo — pack-layer story in four scenes
Ships `core demo audit-tour` as the first investor-facing
walkthrough of the ADR-0027→0041 pack-layer architecture.  Four
scenes, each making one falsifiable claim no transformer-LLM
wrapper can reproduce:

  S1. Identity is geometric, not prompt-veneer.
      Three identity packs load three structurally distinct
      manifolds (ADR-0027).  Distinct alignment thresholds +
      distinct hedge phrases from JSON pack files, not prompts.

  S2. Safety is the universal floor.
      Runtime-checkable safety violation produces a deterministic
      typed refusal string (ADR-0036).  walk_surface preserved
      for audit.  Byte-identical across runs.

  S3. Ethics commitments choose their remediation.
      Per-commitment opt-in (ADR-0037 / ADR-0038): pure-helper
      evidence (should_inject_hedge + inject_hedge worked
      example) against a synthetic violation.  Default pack
      returns False; deployment pack (with acknowledge_uncertainty
      in hedge_commitments) returns True.  Pack JSON drives the
      policy tier.

  S4. Deterministic replay across runtime instances.
      Two fresh ChatRuntime instances, same input, same packs.
      Byte-identical JSONL audit lines (ADR-0040).

Load-bearing evidence over surface inspection: the draft compared
response.surface across packs.  Cold-start hits stub path; pack
differences don't manifest at the surface by design.  Shipped
version pulls evidence from structural surfaces (manifold fields,
opt-in lists, pure helpers) — what actually distinguishes the
packs.  No fake claims.

Scene 3 uses synthetic verdict (not chat()) because ADR-0038
specifies stub path skips hedge by design.  Main-path end-to-end
is asserted in tests/test_hedge_injection.py and referenced in
the tour's evidence comment.

Test gate: tests/test_audit_tour.py asserts
result["all_claims_supported"] is True.  Any scene flipping to
False fails the test and catches the regression.

CLI integration:
  core demo audit-tour          # narration to stdout
  core demo audit-tour --json   # structured report, no narration

Files:
- evals/audit_tour/__init__.py + run_tour.py (new) — 4-scene tour
- core/cli.py — audit-tour target on demo subcommand;
  _AUDIT_TOUR_PREAMBLE; --json suppresses narration
- tests/test_audit_tour.py (new) — 8 tests gating all four claims
- docs/decisions/ADR-0042-audit-tour-demo.md (new) — decision record
- docs/decisions/README.md — ADR index now lists ADR-0027..0042
  + Pack-Layer chain section describing the three-tier composition,
  remediation tiers, and verification surface
- docs/PROGRESS.md — adds core demo audit-tour to verify cheatsheet
- README.md — adds core demo audit-tour to commands cheatsheet

Verification:
- Combined pack-layer + telemetry + tour suite: 220 green
  (was 212 after ADR-0041; +8)
- CLI suites unchanged: smoke 67, runtime 19, cognition 121
- core eval cognition: intent 100%, versor_closure 100% (baseline)
- Manual: core demo audit-tour and --json both correct;
  all_claims_supported = true
2026-05-17 22:06:45 -07:00
Shay
417f71917c feat(adr-0041): core chat --show-verdicts + FanOutSink
Two thin layers closing the audit story end-to-end:

- core chat --show-verdicts prints format_verdict_summary(verdicts)
  to stderr after each turn.  Stdout stays clean for piped
  consumers.  Format is dense and terse; designed to skim, not
  machine-parseable (the JSONL sink owns that contract).

- FanOutSink forwards every emitted line to N sinks in declaration
  order.  Fail-fast on first error — consistent with ADR-0040's
  single-sink contract (audit failures surface).  Composes with
  any combination of JsonlFileSink / JsonlBufferSink / future
  sinks.

Two formatters, one bundle: format_turn_event_jsonl (machine,
ADR-0040) and format_verdict_summary (operator, ADR-0041) both
consume the same TurnVerdicts.  No risk of drift.

Summary format:
  [identity=0.83 safety=ok ethics=VIOLATED:foo refusal=- hedge=YES]

Audit story now reads end-to-end:
  - TurnVerdicts bundle (ADR-0039)
  - Machine JSONL sink (ADR-0040)
  - Fan-out + operator CLI (ADR-0041)

Files:
- chat/telemetry.py — FanOutSink dataclass, format_verdict_summary,
  _format_verdict_short helper
- core/cli.py — --show-verdicts on chat subparser; cmd_chat prints
  summary to stderr when set
- tests/test_telemetry_fanout_and_summary.py (new) — 13 tests
- docs/decisions/ADR-0041-cli-verdicts-and-fanout.md (new)

Verification:
- Combined pack-layer + telemetry suite: 212 green (was 199; +13)
- CLI suites unchanged: smoke 67, runtime 19, cognition 121
- core eval cognition: intent 100%, versor_closure 100% (baseline)
- Manual smoke: echo "light is" | core chat --show-verdicts prints
  expected bracketed audit line to stderr alongside response.
2026-05-17 21:47:47 -07:00
Shay
226f14a941 feat(adr-0040): structured-logging sink for turn-event audit
Adds the canonical JSONL sink surface consuming TurnEvent records
that ADR-0039 made uniform across main and stub paths.  One
deterministic line per turn; redact-by-default trust boundary;
opt-in content emission; runtime auto-emits on attached sink.

Trust boundary (CLAUDE.md):
- Metadata-only by default — no surfaces or input tokens emitted.
  include_content=True opt-in at attachment time.
- Path fixed at construction for JsonlFileSink; no user-controlled
  paths interpreted at emit time.
- Sink errors propagate — telemetry failures should surface, not
  silently drop audit signal.

Determinism:
- sort_keys=True; compact separators. Same event → byte-identical line.
- No implicit wall-clock; timestamps caller-provided.
- Field set fixed; missing TurnEvent attrs fall back to safe defaults.

API:
- serialize_turn_event(event, **kwargs) -> dict  (pure)
- format_turn_event_jsonl(event, **kwargs) -> str (pure, deterministic)
- TurnEventSink Protocol; JsonlBufferSink; JsonlFileSink
- ChatRuntime.attach_telemetry_sink(sink, *, include_content=False)
- _emit_turn_event invoked after both turn_log.append sites

Wire format (alphabetised, always present): cycle_cost_total,
dialogue_role, ethics_pack_id, ethics_runtime_checkable_count,
ethics_upheld, ethics_violated, flagged, hedge_injected,
identity_pack_id, refusal_emitted, safety_pack_id,
safety_runtime_checkable_count, safety_upheld, safety_violated,
stub_path, turn, vault_hits, versor_condition.

Conditional: identity_* (when score present), surface /
walk_surface / articulation_surface / input_tokens (when
include_content=True), timestamp (when provided).

Files:
- chat/telemetry.py (new) — serializer, formatter, sinks
- chat/runtime.py — attach + emit + post-append calls
- tests/test_telemetry_sink.py (new) — 29 tests
- docs/decisions/ADR-0040-telemetry-sink.md (new)

Verification:
- Combined pack-layer + telemetry suite: 199 green (was 170 after
  ADR-0039; +29)
- CLI suites unchanged: smoke 67, runtime 19, cognition 121
- core eval cognition: intent 100%, versor_closure 100% (baseline)
2026-05-17 21:39:58 -07:00
Shay
f3cc408f82 feat(adr-0039): audit completeness — TurnVerdicts bundle, stub TurnEvent, hedge_injected
Closes three audit gaps left by the ADR-0035→ADR-0038 pack-layer
surface:

1. TurnVerdicts bundle (chat/verdicts.py) — frozen dataclass
   aggregating identity_score + safety_verdict + ethics_verdict +
   refusal_emitted + hedge_injected.  Attached to both
   ChatResponse.verdicts and TurnEvent.verdicts.  Fields typed as
   object for the same module-coupling reason as
   TurnEvent.safety_verdict.

2. Stub-path TurnEvent emission — _stub_response accepts optional
   tokens kwarg and appends a TurnEvent to turn_log when invoked
   from a real turn.  Audit consumers can now iterate turn_log
   end-to-end without missing stub paths.  Defensive call sites
   (correct() fallback) bypass the append by omitting tokens.

3. refusal_emitted / hedge_injected flags — runtime tracks whether
   it actually mutated the surface this turn.  hedge_injected uses
   idempotent-on-prefix semantics (True iff the runtime ADDED a
   hedge, not iff a hedge happens to be present).

Test-pattern note: previous "gate on rt.turn_log to detect main vs
stub" pattern is now broken; updated to gate on walk_surface ==
_UNKNOWN_DOMAIN_SURFACE.  One existing hedge-injection test gate
updated accordingly.

Back-compat: ADR-0035→0038 per-field accessors
(response.safety_verdict, etc.) still work.  New consumers should
read response.verdicts.

Files:
- chat/verdicts.py (new) — TurnVerdicts dataclass
- chat/runtime.py — _stub_response tokens kwarg + stub TurnEvent
  append + hedge_injected tracking + bundle construction
- core/physics/identity.py — TurnEvent.verdicts: object = None
- tests/test_turn_verdicts_bundle.py (new) — 16 tests
- tests/test_hedge_injection.py — gate fix for stub detection
- docs/decisions/ADR-0039-audit-completeness.md (new)

Verification:
- Combined pack-layer suite: 170 green (was 154 after ADR-0038)
- CLI suites unchanged: smoke 67, runtime 19, cognition 121
- core eval cognition: intent 100%, versor_closure 100% (baseline)
2026-05-17 21:32:46 -07:00
Shay
ad8495d777 feat(adr-0037,adr-0038): per-predicate ethics refusal + hedge injection
Two sibling escalation tiers above the audit-only ethics baseline,
both opt-in per commitment via the ethics pack JSON.

ADR-0037 — refusal_commitments

- EthicsPack.refusal_commitments (frozenset[str]; subset of
  commitment_ids; validated at load time, unknown id rejected)
- Generic refusal prefix: "I cannot proceed — boundary violated: "
- Source-tagged refusal ids: "safety:<id>" / "ethics:<id>"
- build_refusal_surface now takes (safety_verdict, ethics_verdict,
  ethics_pack); ADR-0036 single-arg call remains valid back-compat
- Default pack ships refusal_commitments: [] — audit-only floor
  preserved
- Re-ratified default pack (mastery sha changes with schema field)

ADR-0038 — hedge_commitments

- EthicsPack.hedge_commitments (sibling field; same validator)
- Mutually exclusive with refusal_commitments at load time
- Runtime prepends manifold's preferred_hedge_soft (fallback
  preferred_hedge_strong) when an opted-in commitment fires
  runtime-checkable
- Refusal supersedes hedge globally; stub path skips hedge (already
  a disclosure surface); main path only
- Idempotent on prefix (case-insensitive) — defends against
  ADR-0028 assembler hedges
- Does NOT flip _last_refusal_was_typed — hedge is not refusal

Surface contract:
- ChatResponse.walk_surface + articulation_surface preserved unchanged
  on both refusal and hedge paths (same audit discipline as ADR-0036)
- Only user-facing ChatResponse.surface (and TurnEvent.surface on
  main path) is mutated

Files:
- packs/ethics/loader.py — refusal_commitments + hedge_commitments
  fields; _validate_opt_in_subset; mutual-exclusion check
- packs/ethics/default_general_ethics_v1.json — both opt-in lists
  empty; re-ratified
- chat/refusal.py — generic prefix, source-tagged ids,
  violated_runtime_checkable_ethics, should_inject_hedge,
  build_hedge_prefix, inject_hedge
- chat/runtime.py — passes ethics_verdict + ethics_pack to refusal
  builder; hedge injection branch after refusal check
- tests/test_ethics_refusal_opt_in.py (new) — 16 tests
- tests/test_hedge_injection.py (new) — 22 tests
- docs/decisions/ADR-0037-per-predicate-ethics-refusal.md (new)
- docs/decisions/ADR-0038-hedge-injection.md (new)

Verification:
- Combined pack-layer suite: 154 green (was 116 after ADR-0036)
- CLI suites unchanged: smoke 67, runtime 19, cognition 121
- core eval cognition: intent 100%, versor_closure 100% (baseline)
2026-05-17 21:23:28 -07:00
Shay
a0372c951f feat(adr-0036): safety-only typed refusal policy
Runtime-checkable SafetyVerdict violations now replace
ChatResponse.surface (and TurnEvent.surface on the main path) with a
deterministic typed refusal string.  Ethics violations remain
audit-only.

Why safety-only: safety is the universal floor (ADR-0029,
never-swappable, fail-closed).  Ethics is swappable per-deployment;
wiring ethics into refusal would let pack-swappers silently change
refusal behavior via JSON edit.  Wrong coupling.

Why typed refusal (not hedge injection / not re-articulation): typed
refusal is deterministic, audit-detectable by prefix, and preserves
replayability.  Hedge injection would blur surface-preferences-driven
hedging vs predicate-driven refusal.  Re-articulation retry yields the
same surface (planner is deterministic; no refusal-bias hint surface
exists).  Deferred to a future ADR.

Refusal contract:
- ChatResponse.surface = typed refusal string
- walk_surface + articulation_surface = unchanged (audit preserved)
- runtime._last_refusal_was_typed = True (next-turn evidence for
  no_silent_correction)
- Only runtime_checkable=True violations refuse
- Stub path symmetric

Files:
- chat/refusal.py (new) — pure refusal builder + audit helpers
- chat/runtime.py — invoke build_refusal_surface after safety_verdict
- tests/test_safety_refusal.py (new) — 20 tests
- docs/decisions/ADR-0036-safety-refusal-policy.md (new)

Verification:
- 20 new tests; combined pack-layer suite 116 green
- CLI suites unchanged: smoke 67, runtime 19, cognition 121
- core eval cognition: intent 100%, versor_closure 100% (baseline)
2026-05-17 21:10:52 -07:00
Shay
514ace0cbf feat(adr-0035): turn-loop auto-invocation — surfacing only
Wires SafetyCheck and EthicsCheck into ChatRuntime at end-of-turn on
both the main articulation path and _stub_response.  Verdicts attach
to ChatResponse.safety_verdict / .ethics_verdict and TurnEvent.
Observational at v1: no refusal, no re-articulation, no behavioral
change.  Refusal policy is the next ADR with real verdict data in hand.

Runtime-checkable predicates today:
  - preserve_versor_closure         (via _FieldStateWithVersor adapter)
  - no_identity_override            (manifold hash before vs after; equal by construction)
  - no_silent_correction            (runtime._last_refusal_was_typed bookkeeping)
  - acknowledge_uncertainty         (IdentityScore.alignment + hedge detection)
  - disclose_limitations            (walk_surface == _UNKNOWN_DOMAIN_SURFACE)

Predicates with no runtime evidence (no_manipulation, no_fabricated_source,
defer_high_stakes_to_human_review, respect_user_autonomy, no_hot_path_repair)
honestly report runtime_checkable=False per the ADR-0032/0034 discipline.
They become checkable as classifiers and pipelines land — surface contract
doesn't change.

Test coverage: 14 new tests; combined pack-layer surface suite (loaders +
checks + turn-loop) now 122 green.  CLI suites unaffected: smoke 67,
cognition 121, teaching 17, runtime 19.  Cognition eval baseline preserved.
2026-05-17 20:57:33 -07:00
Shay
db5bc028f9 feat(adr-0034): EthicsCheck — structural surface parallel to SafetyCheck
Completes the predicate-surface layer for ethics packs, sibling to
ADR-0032's SafetyCheck.  Same registry-of-predicates shape; same
observational discipline; same honest reporting of runtime-checkable=False
for structural commitments that cannot be evaluated from per-turn evidence.

Five default predicates for the v1 commitments:

  acknowledge_uncertainty           — alignment < threshold ⇒ requires hedge
  defer_high_stakes_to_human_review — high_stakes ⇒ requires recommend_review
  disclose_limitations              — ungrounded ⇒ requires disclosure marker
  no_manipulation                   — structural; runtime_checkable=False
  respect_user_autonomy             — prescriptive ⇒ requires ≥2 options surfaced

`no_manipulation` is the ethics-side analogue of `no_hot_path_repair`
in SafetyCheck — an aggregate property enforced by realizer design and
review, not a per-turn metric.  Honest reporting rather than a silent
upheld pass.

ChatRuntime exposes `runtime.ethics_check`; turn loop does not
auto-invoke.  Refusal / re-articulation wiring is a future ADR.

Test coverage: 27 new tests; combined pack-layer surface suite
(identity + safety + ethics, loaders + checks) is now 108 tests, all
green.  Cognition (121), teaching (17), runtime (19), smoke (67)
unaffected.
2026-05-17 20:46:34 -07:00
Shay
dab7b9c061 feat(adr-0033): ethics packs — third pack-layer sibling to identity + safety
Completes the three-layer pack architecture:
  identity (who CORE is)  + safety (universal red lines)
                          + ethics (deployment-specific propositional commitments)

  manifold.boundary_ids = identity.boundary_ids
                        ∪ safety.boundary_ids
                        ∪ ethics.commitment_ids

Ethics packs are swappable like identity (fall back to default on load
failure) but propositional like safety (commitment ids union into the
manifold).  EthicsPackError inherits from ValueError; only when both
the requested and default packs fail does startup refuse.

Ships default_general_ethics_v1 with five commitments:
  - acknowledge_uncertainty
  - defer_high_stakes_to_human_review
  - disclose_limitations
  - no_manipulation
  - respect_user_autonomy

Ratified through identity_anchor template at SHA 81fc9b61c828….

Test coverage: 20 new tests; combined identity/safety/ethics surface
suite is 81 tests, all green.  Cognition (121), teaching (17), runtime
(19), smoke (67), and cognition eval all unaffected.
2026-05-17 20:41:04 -07:00
Shay
6f67e9a616 feat(safety): ADR-0032 — SafetyCheck structural surface
Closes the 'boundaries are checked at scattered call sites' gap noted
in ADR-0029.  Adds a centralized observational surface parallel in
shape to IdentityCheck — produces a verdict, does not refuse.  Wiring
verdicts into refusal paths is a future ADR.

Shape (parallel to IdentityCheck, different in mechanism):

  SafetyContext     — duck-typed input bag (field_state, citations,
                       refusal-was-typed flag, identity manifold hashes
                       before/after).  Every field optional with safe
                       defaults; absence of evidence is not evidence of
                       violation.
  SafetyCheckResult — per-boundary: boundary_id, upheld, reason,
                       runtime_checkable, evidence tuple.
  SafetyVerdict     — aggregate: pack_id, results (lex order on
                       boundary_id), upheld, violated_boundaries,
                       runtime_checkable_count.
  SafetyCheck       — registry of predicates; check(ctx, pack) returns
                       SafetyVerdict.  register(boundary_id, predicate)
                       adds custom predicates.

Five default predicates for v1 boundaries:

  preserve_versor_closure   runtime_checkable=True   field.versor_condition < 1e-6
  no_fabricated_source      runtime_checkable=True*  cited ⊆ allowed
  no_silent_correction      runtime_checkable=True   last refusal was typed
  no_identity_override      runtime_checkable=True*  hash before == hash after
  no_hot_path_repair        runtime_checkable=FALSE  code-path; static-analysis

  *Conditional on the caller supplying the necessary fields.

The honest answer on no_hot_path_repair: it is a code-path boundary
enforced by static analysis + code review.  Runtime cannot judge it.
A predicate that silently reported upheld=True would be a small lie —
exactly the kind of thing CLAUDE.md forbids.  SafetyCheck reports
runtime_checkable=False with a clear reason so auditors see the truth.

ChatRuntime integration:
  ChatRuntime.__init__ now constructs self.safety_check = SafetyCheck()
  alongside self._identity_check.  Turn loop does NOT auto-invoke at
  v1 — operators and future ADRs decide when/where to call it.

Files:
  packs/safety/check.py            new — SafetyCheck + value types +
                                   default predicates
  packs/safety/__init__.py         re-exports the new public surface
  chat/runtime.py                  constructs self.safety_check
  tests/test_safety_check.py       new — 20 tests covering each
                                   default predicate (positive +
                                   negative), unknown-boundary
                                   fallback, custom registration,
                                   defensive boundary-id rebinding,
                                   verdict aggregation, ChatRuntime
                                   integration
  docs/decisions/ADR-0032-safety-check-surface.md  Accepted
  docs/safety_packs.md             §SafetyCheck section added,
                                   known-limit #1 struck through
  memory/safety-pack.md            refreshed; new follow-up about
                                   turn-loop auto-invocation

Suite status (all green):
  cognition 121, teaching 17, runtime 19, formation 182, smoke 67
  identity / safety / surface divergence suites: 108 tests passing
  (was 88 before this ADR; +20 safety-check tests)

Scope limits (documented):
  - No auto-invocation in the turn loop.
  - No refusal wiring on violation.
  - No refactoring of existing scattered enforcement sites.
  - Defensive boundary-id rebinding masks predicate bugs; debug-mode
    surfacing is a future enhancement.
2026-05-17 20:25:22 -07:00
Shay
07ad3af845 feat(surface): ADR-0031 — score-decomposition surface (per-axis hedges)
Closes the 'identity hedges are generic' gap.  When IdentityCheck reports
that a specific axis is deviating AND the pack supplies an axis_hedges
entry for that axis, the assembler uses that axis's phrase instead of
ADR-0028's generic preferred_hedge_*.  The hedge text now names what is
actually at issue.

Selection: lex-smallest axis_id in (ctx.deviation_axes ∩ axis_hedges).
Deterministic; loader emits axis_hedges in lex order on axis_id.

Example surface at alignment=0.30 (strong band) under default pack:
  No deviation             → 'It seems that truth reveals reality.'
  truthfulness deviates    → 'Evidence is thin that truth reveals reality.'
  coherence deviates       → 'This does not yet cohere: truth reveals reality.'
  reverence deviates       → 'Reports suggest truth reveals reality.'

Same trajectory + truthfulness deviation, three different packs:
  default_general_v1   → 'Evidence is thin that truth reveals reality.'
  precision_first_v1   → 'The evidence does not support that truth reveals reality.'
  generosity_first_v1  → 'Truth reveals reality.'  (above generosity's strong=0.20)

Schema (additive, optional):
  surface_preferences.axis_hedges = {
    <axis_id>: { 'strong': str, 'soft': str, 'qualifier': str },
    ...
  }

Bounds: each phrase length 1–64; axis_id non-empty.  Absent block →
ADR-0028 byte-for-byte fallback.  Loader emits pairs in lex order on
axis_id for hashability + deterministic tie-break.

Files:
  core/physics/identity.py
    + class AxisHedge (frozen: strong, soft, qualifier)
    SurfacePreferences gains axis_hedges: Tuple = ()
  packs/identity/loader.py
    + _build_axis_hedges(): parse + bounds-check + emit lex-ordered tuple
  generate/surface.py
    SurfaceContext gains deviation_axes: frozenset[str] + axis_hedges tuple
    + _axis_specific_phrase(ctx): lex-smallest match or None
    _apply_hedge consults axis-specific phrase before ADR-0028 fallback
    Depth languages (he, grc) unchanged — ADR-0030 canonical phrases
  chat/runtime.py
    _build_surface_context lifts identity_score.deviation_axes and
    prefs.axis_hedges into SurfaceContext
  packs/identity/*.json
    Three v1 packs gain axis_hedges blocks (truthfulness, coherence,
    reverence — each pack uses voice consistent with its character)
  scripts/ratify_identity_packs.py (no change — idempotent)
  packs/identity/*.mastery_report.json
    Auto-refreshed.  New SHAs:
      default_general_v1   → 2ab7d469013509ba5030313ca9a609a443d0716e3ddcc5596f59858ce054f5d3
      precision_first_v1   → 78aa1e6a68a35c2c8576b6196a52d421b94f6d11e006128986902a4fd08679af
      generosity_first_v1  → 511f1ce20edd4266239da61443bfc93473a5433f20bfee6692a25a03073dc933

Tests: tests/test_identity_score_decomposition.py — 17 new tests:
  per-axis phrase selection, band gating still applies, pack swap with
  same deviation produces three different phrases, lex tie-break is
  deterministic, depth-language fallback to ADR-0030, backward compat
  with empty deviation_axes, and the contract that all three v1 packs
  ship axis_hedges for all three default-pack axes.

Suite status (all green):
  cognition 121, teaching 17, runtime 19, formation 182, smoke 67
  identity+safety+English+depth divergence 71
  score decomposition 17

Scope limits (documented in ADR-0031):
  - English-only at v1 (depth languages use canonical ADR-0030 phrases)
  - Lex tie-break is operational not semantic — pack authors can re-key
    if they need a different priority
  - No dominance-driven phrasing (Interpretation A); preserved as
    forward-compatible follow-up

Docs: ADR-0031 (Accepted) recorded; docs/identity_packs.md gains
§Axis-specific hedge phrases section and updated v1-pack SHAs; memory
'identity-packs.md' refreshed.
2026-05-17 20:16:22 -07:00
Shay
a49a7555dc feat(surface): ADR-0030 — depth-language hedge wiring
Closes the ADR-0028 'English-only differentiation' gap.  Hebrew and
Koine Greek surfaces now consult identity-pack surface_preferences for
hedge and claim-strength shaping, using language-appropriate canonical
hedge phrases.  CORE's three-language foundation (English / Hebrew /
Greek) is now uniformly identity-aware at the realizer.

Algorithm: the same four-band hedge/claim-strength logic from ADR-0028
runs for all three languages.  Thresholds and claim_strength come from
the identity pack (carried on SurfaceContext).  Hedge phrases come
from ctx for English and from a new module-level constant
_DEPTH_HEDGE_PHRASES for Hebrew (he) and Koine Greek (grc).

  he:  'נראה ש' / 'אולי' / 'במקרים מסוימים,'
  grc: 'δοκεῖ ὅτι' / 'ἴσως' / 'ἐνίοτε,'

Pack swap visibly affects depth-language output: a precision_first
identity pulls hedges to higher alignment than default; a generosity
pack pulls them to lower alignment.  Same trajectory through the
manifold → three different Hebrew surfaces under three different
packs.  Same for Greek.

Files:
  generate/surface.py
    _DEPTH_HEDGE_PHRASES (new module constant)
    _apply_hedge(surface, ctx, lang='en')   — lang param added
    _assemble_he(.., ctx)                   — ctx param added
    _assemble_grc(.., ctx)                  — ctx param added
    SentenceAssembler.assemble              — passes context to he/grc
  tests/test_identity_surface_divergence_depth.py — 15 new tests:
    Hebrew hedge bands, Greek hedge bands, pack-swap divergence in
    both depth languages, three-language hedge phrase distinctness,
    backward compatibility with ctx=None
  docs/decisions/ADR-0030-depth-language-hedge.md  — Accepted
  docs/identity_packs.md                            — closes known-limit #1
  memory/identity-packs.md                          — refreshed

Backward compat:
  - _apply_hedge default lang='en' so existing callers unaffected.
  - English surface output byte-for-byte unchanged.
  - _assemble_he / _assemble_grc with ctx=None match pre-ADR output
    byte-for-byte (asserted by TestBackwardCompatibility).

Scope limits (documented in ADR):
  - Depth-language hedge phrases are canonical defaults, not per-pack
    overridable yet.  Future ADR may add a 'languages' block to the
    pack schema if a downstream deployment needs override capability.
  - Contrast ('However, ...') and subordination ('Given that ..., ...')
    remain English-only.  Hedge is the dominant differentiator.
  - Hebrew/Greek grammar / word order unchanged.

Suite status: cognition 121, teaching 17, runtime 19, formation 182,
smoke 67 — all green.  Identity + safety + divergence suites: 26+15+15+15=71
all green.
2026-05-17 20:05:45 -07:00
Shay
ece73c76d5 feat(safety): ADR-0029 — always-loaded, never-replaceable safety pack
Closes the trust gap ADR-0027 opened: making the identity manifold
swappable was necessary for downstream robotics / personalization /
creative deployments, but it left nothing structurally preventing a
downstream identity pack from disabling core safety constraints.
Safety packs sit at a separate trust layer, fail closed on every error
path, and union their boundaries into every runtime manifold regardless
of which identity pack is selected.

Architecture (sibling to identity packs, structurally distinct):

  Layer            Swappable?  Removable?  Schema
  ---------------  ----------  ----------  -----------------------------
  Safety pack      No          No          boundary_ids + descriptions
  Identity pack    Yes         No          value_axes + surface_prefs
  Language pack    Yes         (>=1 reqd)  vocab / morphology / packs

Composition rule (at ChatRuntime startup, additive only):

  identity = load_identity_manifold(config.identity_pack)
  safety   = load_safety_pack()                        # fail-closed
  final.boundary_ids = identity.boundary_ids ∪ safety.boundary_ids

Safety contributes boundaries only — no value_axes, threshold, or
surface_preferences.  This keeps existing tests that assert on identity
axis sets passing byte-for-byte, and matches the semantic intent
(safety is what's forbidden, not what's pulled toward).

Shipping safety pack: packs/safety/core_safety_axes_v1.json
  → mastery_report_sha256 ee1249acdf8c273aeb656d803c37ef915e536d85f177f5cc18c6e2f6c995ce29

Five v1 boundaries, each closing a specific CLAUDE.md doctrine:
  no_fabricated_source       — no invented provenance
  no_hot_path_repair         — no normalization in propagate/stream/store
  no_identity_override       — user text cannot mutate identity
  no_silent_correction       — failures are typed and visible
  preserve_versor_closure    — ||F * reverse(F) - 1||_F < 1e-6

Fail-closed semantics:
  SafetyPackError inherits from RuntimeError (NOT ValueError) so
  catch-and-continue is discouraged at the type level.  Missing file /
  malformed JSON / empty boundaries / duplicate boundary / failed
  self-seal all raise.  ChatRuntime.__init__ does not catch.

Files:
  packs/safety/core_safety_axes_v1.json              shipping pack
  packs/safety/core_safety_axes_v1.mastery_report.json  signed report
  packs/safety/__init__.py                           public surface
  packs/safety/loader.py                             load_safety_pack(),
                                                     SafetyPack,
                                                     SafetyPackError,
                                                     DEFAULT_SAFETY_PACK
  scripts/ratify_safety_pack.py                      idempotent driver
  chat/runtime.py                                    composition wiring
  tests/test_safety_pack.py                          15 tests:
                                                       loader bounds,
                                                       fail-closed,
                                                       composition under
                                                       all 3 identity packs
  docs/decisions/ADR-0029-safety-packs.md            decision record
  docs/safety_packs.md                               operational ref
  README.md                                          §Safety Pack added
  memory/safety-pack.md                              auto-memory entry

Suite status: cognition 121, teaching 17, runtime 19, formation 182,
smoke 67, identity 41, safety 15 — all green.
2026-05-17 19:56:29 -07:00
Shay
7c839d2e12 feat(cli): core chat --list-identity-packs + companion-file filter
Adds the discovery flag callers have been asking for since ADR-0027.
Short-circuits before the REPL launches; supports both a human-readable
table and `--json` machine output.  Drives the loader's existing
`available_packs()` helper.

Bug fix on the way: `available_packs()` was globbing every `*.json`
in the search path, so the Phase-5 companion `<pack_id>.mastery_report.json`
files were leaking into the list as fake packs with empty fields.  The
helper now skips any file ending in `.mastery_report.json` and rejects
JSON that lacks the required `schema_version` / `value_axes` fields.

CLI output:

  pack_id              version  ratified  description
  -------------------  -------  --------  -----------
  default_general_v1   1.0.0    yes       Balanced general identity...
  generosity_first_v1  1.0.0    yes       Generosity-first specialization...
  precision_first_v1   1.0.0    yes       Precision-first specialization...

Tests: +3 (CLI table, CLI JSON, companion-file filter regression).
test_identity_packs.py: 23 -> 26.  cognition / smoke green.

Docs: docs/identity_packs.md CLI usage block updated; memory
'identity-packs.md' closes that follow-up.
2026-05-17 19:47:13 -07:00
Shay
1574a4b030 feat(identity-packs): ADR-0028 — pack-driven hedge & claim-strength shaping
Closes the 'identity is load-bearing but not visibly differentiated'
gap noted at the end of ADR-0027.  Pack swap now produces visibly
different surfaces on identical trajectories at the same alignment.

Schema bump — packs gain an optional 'surface_preferences' block:

  hedge_threshold_strong, hedge_threshold_soft  → band entries
  preferred_hedge_strong, preferred_hedge_soft  → phrases per band
  claim_strength                                → balanced|qualified|affirmative
  qualified_band_high, preferred_qualifier      → marginal-band shaping

Loader enforces threshold ordering (strong <= soft <= qual_high),
phrase length bounds, and the enum-of-three for claim_strength.
Missing block resolves to defaults that reproduce pre-ADR behavior
byte-for-byte; existing tests pass unchanged.

Algorithm (deterministic, surface-only, no sampling/repair/normalize):

  alignment < strong              → preferred_hedge_strong + lower-cased surface
  alignment < soft                → preferred_hedge_soft + lower-cased surface
  soft <= alignment < qual_high
    and claim_strength=qualified  → preferred_qualifier + lower-cased surface
  otherwise                       → bare surface

Three v1 pack profiles:

  default_general_v1   balanced; 0.40 / 0.50 / 0.75 ; 'It seems that' / 'Perhaps'
  precision_first_v1   qualified; 0.55 / 0.70 / 0.85 ; 'Arguably,' / 'In some cases,' / 'Under certain conditions,'
  generosity_first_v1  affirmative; 0.20 / 0.30 / 0.50 ; default hedge phrases

Re-ratified.  New MasteryReport SHAs (superseding Phase-5):

  default_general_v1   → ddc1ba127231272660e6a435e177227558461b0278572a95635b416c3e1dec5a
  precision_first_v1   → cb5fb2323214a26afda33f2a67e22f38fe49f4763829d48ef67fd41241aba33c
  generosity_first_v1  → 94f2f49e1b16c7498fb52b8f9864eecc198618933dc8381a01b809c146826db7

Files touched:

* core/physics/identity.py — new SurfacePreferences dataclass;
  IdentityManifold gains 'surface_preferences' field with defaults.
* packs/identity/loader.py — _build_surface_preferences() parses,
  bounds-checks (threshold ordering, claim_strength enum, phrase
  length, threshold ranges); SurfacePreferences round-trips.
* generate/surface.py — SurfaceContext gains 7 new fields with defaults
  matching the pre-ADR module-level HEDGE_STRONG_THRESHOLD /
  HEDGE_SOFT_THRESHOLD; _apply_hedge takes the full context and
  implements the four-band algorithm; module-level constants retained
  for back-compat.
* chat/runtime.py — _build_surface_context lifts manifold.surface_preferences
  into SurfaceContext.
* packs/identity/*.json — three v1 packs gain surface_preferences blocks
  tuned to their roles; re-ratified via scripts/ratify_identity_packs.py
  (idempotent).
* tests/test_identity_surface_divergence.py — 15 tests covering hedge
  bands, claim_strength bands, pack-swap divergence proof, and runtime
  context wiring.

Suite status: cognition 121, teaching 17, runtime 19, formation 182,
smoke 67 — all green.  test_identity_packs.py 23/23, new
test_identity_surface_divergence.py 15/15.

Docs: ADR-0028 (Accepted) records the decision and verification; ADR-0027
status updated to point to ADR-0028 for deep realizer wiring; README
§Identity Packs notes the visible divergence; docs/identity_packs.md
gains a §Surface preferences section and closes the known-limit #1
about invisible surface differentiation.
2026-05-17 19:42:54 -07:00
Shay
c3e36f07b2 feat(identity-packs): ADR-0027 Phase 5 — ratify all three v1 packs
Drives the three v1 identity packs through the full formation pipeline
(Forge -> Compose -> Compile -> Run -> Ratify) and embeds the resulting
self-sealed MasteryReport SHAs into each pack file.  Companion
'<pack_id>.mastery_report.json' artifacts ship alongside.  Loader now
defaults to production mode (require_ratified=None) and ChatRuntime
calls it without the dev-only override.

Ratification results:
  default_general_v1   -> 0b77357fe4359f161d7ca72f184b6e0db2f9e2de16b32c237a3b80d2bbb005b4
  precision_first_v1   -> 5f5000dba9a0dd19d831e9ab5d3c0e3b9faf6abdc2648940e96aa6263af3302e
  generosity_first_v1  -> 91716117558113f74b2c6d07a804cb324f262d62b743523d901d1386a4f85ae4

Driver: scripts/ratify_identity_packs.py — idempotent.  Re-running on
already-current packs is a no-op (verified by a test).  Each pack is
treated as its own provenance source: source_sha = SHA-256 of the pack's
canonical JSON body with mastery_report_sha256 blanked, so the
self-referential chain stays stable across SHA updates.  Axes become
ConceptCandidates; canned override-attempt triples become
CounterCandidates; the identity_anchor template renders the body.

Loader hardening (packs/identity/loader.py):
  * When require_ratified resolves to True, the loader now requires the
    companion '<pack_id>.mastery_report.json' to exist, its
    report_sha256 to match the pack's mastery_report_sha256, and its
    self-seal to verify via formation.hashing.verify_seal.
  * Tampered companion (wrong SHA, broken seal) is rejected with a
    diagnostic IdentityPackError.

Tests: 18 -> 23.  New cases cover production-mode loading of all three
v1 packs, missing companion file, mismatched companion SHA, failed
self-seal, and end-to-end idempotency of the ratification script
(subprocess-launched, asserts pack bytes unchanged on re-run).

Suite status: cognition 121, teaching 17, runtime 19, formation 182,
smoke 67 — all green.

Docs updated: ADR-0027 status flipped to Phases 1-6 complete with the
three report SHAs recorded; docs/identity_packs.md notes the ratified
SHAs and the re-ratification command; memory file 'identity-packs.md'
refreshed.
2026-05-17 19:31:55 -07:00
Shay
fa05be9293 feat(identity-packs): ADR-0027 — swappable identity manifold via packs
Replaces the hardcoded IdentityManifold constructor in chat/runtime.py
with a content-addressed pack loader.  Identity is now load-bearing AND
swappable: deployments select an identity pack at startup, downstream
builders (robotics, personalization, creative tools) author their own
ratified packs without editing CORE Python.

Phase 1 — pack format + loader
  * packs/identity/loader.py — load_identity_manifold(pack_id, *,
    search_paths, require_ratified) with bounds checks (axis count,
    direction in [-1, 1], weight in [0, 10], threshold in [0, 1],
    axis-id uniqueness).
  * available_packs() helper for discovery.
  * IdentityPackError raised on every bounds violation.

Phase 2 — three v1 packs
  * default_general_v1.json — ship default; encodes the previous
    hardcoded three axes (truthfulness, coherence, reverence)
    byte-for-byte so existing runtime behavior is preserved.
  * precision_first_v1.json — boosts truthfulness weight, narrows
    coherence/reverence; tighter alignment threshold.
  * generosity_first_v1.json — boosts coherence weight, broadens
    reverence; looser alignment threshold.

Phase 3 — replace hardcoded constructor
  * chat/runtime.py:206 calls load_identity_manifold() using
    RuntimeConfig.identity_pack (default DEFAULT_IDENTITY_PACK).
  * Dead _default_identity_manifold() removed.
  * ChatRuntime.identity_pack_id surfaces the loaded pack id.

Phase 4 — CLI flag
  * core chat --identity <pack_id>  (also threaded into trace/oov via
    _add_runtime_policy_args).
  * core/config.py: RuntimeConfig.identity_pack added; empty string
    falls back to DEFAULT_IDENTITY_PACK = 'default_general_v1'.

Phase 5 — formation ratification — INTENTIONALLY DEFERRED.  Loader
currently calls require_ratified=False so the v1 packs (which carry
empty mastery_report_sha256) load.  Authoring SubjectSpecs for each
pack, running the formation pipeline end-to-end to produce signed
MasteryReports, and embedding the SHA into each pack file is a
follow-up.

Tests: 18 new tests in tests/test_identity_packs.py covering loader
happy paths, every bounds violation, runtime wiring, and pack-swap
divergence.

Suite status: cognition 121, teaching 17, runtime 19, formation 182,
smoke 67 — all green.

Docs: ADR-0027 (Accepted) + docs/identity_packs.md (operational ref) +
README.md §Identity Packs + docs/teaching_order.md Layer 1 cross-ref.
2026-05-17 19:24:39 -07:00
Shay
7feb239fdd feat(formation/templates): four new course templates + shared helpers
Adds the four templates called out in docs/teaching_order.md so the formation
pipeline can ratify more than just definitional ontologies:

* composed_relation — Layer 4.  Chains are the unit of mastery; each chain of
  length >= 2 emits a composed_relations entry with composition_kind
  (transitive | lifting), an inferred relation, and chain-break adversarial
  probes drawn from counters or canned.
* procedural — ordered state transitions; strict_linear_topo refuses
  branches, cycles, and disconnected components at render time.
  ordering_hints validated against the linear chain.  Canned violation
  probes for precondition_violation / step_skip / back_edge.
* falsification — counter-example-driven.  Counters move to Phase 2 paired
  with coherent alternatives drawn from relations sharing the same head.
  Unmatched counters surface in unmatched_counters; false-coherent probes
  emitted per pair.
* identity_anchor — Layer 1 seeding.  Concepts interpreted as identity axes
  ranked by ordering_hints; counters interpreted as override attempts;
  canned IDENTITY_OVERRIDE_PROBES always appended.

Common helpers extracted to formation/templates/_common.py: canonical
constants (MAX_VERSOR_CONDITION, RATIFICATION_GATES, PROMOTION_PATH,
IDENTITY_OVERRIDE_PROBES, NORMALIZATION_FORBIDDEN_SITES), deterministic
ordering (sorted_concepts/_counters/_hints, topo_sorted_relations,
strict_linear_topo), payload builders, geometric_dependencies,
maximal_chain_walks, adversarial_block, course_id, subject_payload,
substrate_invariants_payload, phase_5_payload.

formation/templates/__init__.py now dispatches via a lazy-import _REGISTRY
keyed by template_id; registered_template_ids() exposed for callers and
tests.  definition.py refactored to use _common verbatim — byte-stability
preserved (existing test_compose.py still passes; test_sha_stable_across_
subprocess unchanged).

Tests: 44 new tests across test_template_{composed_relation,procedural,
falsification,identity_anchor,registry}.py.  Each new template gets
determinism, paradigm-structure, error-handling, and cross-subprocess SHA
stability tests; registry test asserts the five known ids and that
identical inputs through different templates produce different SHAs.

Formation suite: 138 -> 182 passing.  cognition (121) and smoke (67)
suites unchanged.  ratify.py enforcement of the new paradigm-specific
gates (every_composed_relation_replayed, linear_order_strict, etc.)
remains a documented follow-up — templates declare the gates in their
phase_5 body so the ratifier extension is purely additive.
2026-05-17 18:59:15 -07:00
Shay
717eaf6ed7 docs: teaching-order doctrine + refresh formation/roadmap status
- Add docs/teaching_order.md as durable reference for curriculum ordering.
  Five-layer rule (identity axes -> atomic definitions -> binary relations ->
  composed relations -> domain expansion), grounded in ratify.py G3,
  MasteredCoursesIndex, and exact CGA distance. Linked from README.md.
- Mark formation_pipeline_plan.md as IMPLEMENTED (back half); enumerate the
  open items not closed by Phases 1-7 (additional templates, first formation-
  routed curriculum, G2 activation).
- Add 2026-05-17 status block to capability_roadmap.md covering the FSC chain,
  epistemic schema closure, formation pipeline back half, FSC v3 proof matrix,
  cost benchmark, and the pulse import fix.
2026-05-17 18:48:44 -07:00