Commit graph

172 commits

Author SHA1 Message Date
Shay
7cc2f7b422
feat(adr): ADR-0107 mathematics_logic expert-demo promotion deferred (#114)
The ADR-0106 contract correctly refused promotion. ADR-0107 records the
deferral and reserves two follow-up ADRs:

- ADR-0109 (lane-shape-aware threshold amendment): ADR-0106 \xc2\xa71.2
  prescribes cognition-pack-shape metrics uniformly, but math /
  physics / systems / hebrew-greek lanes carry native shapes
  (accuracy, passed_rate, all_pass_rate). Prerequisite for any future
  expert-demo promotion.
- ADR-0110 (math re-attempt): conditional on ADR-0109 landing and
  inference_closure substantively passing (currently all_pass_rate=0.4
  on public).

tests/test_adr_0107_deferral.py pins adr_0107_no_silent_promotion: math
stays at reasoning-capable, has no expert_demo_claims entry, and the
ledger row carries a named refusal reason.

No change to core/capability/expert_demo.py or reporting.py -- the
contract is honored, not amended. README sequencing updated to reflect
ADR-0107 acceptance and the new ADR-0109/0110 prerequisites.
2026-05-22 11:49:37 -07:00
Shay
0493808215
feat(capability): implement ADR-0106 expert-demo promotion contract (#113)
Closes ADR-0106 acceptance evidence:

- ExpertDemoClaim dataclass + additive expert_demo_claims block on
  ReviewerRegistry (schema_version stays at 1; backward-compatible).
- New core/capability/expert_demo.py with derive_evidence_digest,
  evaluate_expert_demo, collect_domain_lanes, materialise_lane_results.
- core/capability/reporting.py: replaces the cognition-lane-only
  predicate (previous lines 418-433) with a domain-aware,
  reviewer-signed gate; ledger rows now also carry
  expert_demo_reason for operator legibility. Reviewer registry is
  fail-closed: an unloadable registry yields zero claims, so a broken
  registry never silently grants expert_demo=true.
- tests/test_expert_demo_contract.py covers all three ADR-0106
  invariants: requires_signature, domain_aware, replay_byte_equality;
  plus threshold + production-ledger-untouched gates. 12 new tests.
- tests/test_reviewer_registry.py extended with TestExpertDemoClaimsSchema
  covering omitted block, valid parse, unknown signer rejection,
  malformed digest rejection, duplicate domain rejection. 5 new tests.
- README index row + table preface updated to note expert_demo is
  contract-gated. Frontier list trimmed (ADR-0106 has landed).
- ADR-0106 Status flipped Proposed -> Accepted.

No domain row's expert_demo field flips by this PR -- only the contract
changes. Promotion of any ratified domain requires a follow-up ADR
(ADR-0107 reserved for mathematics_logic) plus a signed claim.
2026-05-22 11:39:09 -07:00
Shay
7bb8334fc5
docs: accept ADR-0108 and rewrite Current frontier per its acceptance evidence (#112)
- ADR-0108 Status: Proposed -> Accepted
- README index row updated to Accepted
- 'Current frontier' rewritten with the ranked Proposed-ADR list mandated
  by ADR-0108 \xc2\xa7Decision; removes the now-false 'No ADR currently sits in
  a "Proposed but unimplemented" state' sentence
- Open candidate directions (no-ADR-yet) section retained for the
  multi-reviewer governance frontier item from ADR-0105
2026-05-22 11:21:20 -07:00
Shay
7e50738034
docs: propose ADR-0108 sequencing for currently-Proposed ADRs (#111)
Makes the post-ADR-0105 sequencing of ADR-0080 / 0084 / 0087 / 0106
explicit, durable, and revisable. Status: Proposed. No content of the
four sequenced ADRs is modified — sequencing is meta, not content.
2026-05-22 10:52:44 -07:00
Shay
8e749546b2
docs: propose ADR-0106 expert-demo promotion contract (#109)
Defines a domain-aware, reviewer-signed expert_demo promotion gate to
replace the current cognition-lane-only predicate in
core/capability/reporting.py:418. Status: Proposed. This ADR does not
promote any domain — it defines the contract that a follow-up ADR (likely
mathematics_logic as ADR-0107) will consume as the first worked
promotion.
2026-05-22 10:44:36 -07:00
Shay
257fd4503d
feat(evals): ADR-0105 — sealed holdout encryption via age (#108)
* feat(evals): add pyrage dependency

* feat(evals): add sealed holdout path resolution

* feat(evals): implement sealed holdout decryption

* feat(evals): add sealed holdout CLI

* test(evals): add sealed holdout encryption tests

* docs(decisions): add ADR-0105 sealed holdout encryption

* feat(evals): route holdout split through sealed decryptor

* docs(decisions): add ADR-0105 index entry

* chore: restore project description

* fix(evals): use pyrage Identity.from_str and pin curriculum SHA

- holdout_runner: pyrage exposes Identity.from_str, not from_file; parse
  identity file by line and pass list[Identity] into decrypt(). Restores
  PR 108's sealed-holdout test suite to green.
- verify_lane_shas: realign curriculum_loop_closure pin with the actual
  deterministic runner output (carryover from PR 107).
2026-05-22 10:09:43 -07:00
Shay
f7680e96ea
feat(teaching): ADR-0104 — curriculum-sourced teaching proposals (#107)
* feat(teaching): add curriculum-sourced proposal builder

* test(teaching): cover curriculum proposal construction

* test(evals): add curriculum loop closure contract

* test(evals): add curriculum loop closure runner

* test(evals): add canonical curriculum loop closure report

* ci(lanes): pin curriculum loop closure lane

* docs(adr): add ADR-0104 curriculum sourced proposals

* docs(adr): register ADR-0104 and seven pinned lanes

* docs(teaching): mark curriculum source activation

* fix(ci): pin curriculum_loop_closure SHA to runner output

* fix(ci): register curriculum_loop_closure in CLAIMS.md generator
2026-05-22 10:05:14 -07:00
Shay
1395ec1354
feat(packs): ADR-0103 — attach hebrew_fluency + koine_greek_fluency lanes to ADR-0102 (#106)
* feat(evals): add Hebrew fluency holdout cases

* feat(evals): add Koine Greek fluency holdout cases

* feat(packs): attach fluency lanes to he_core_cognition_v1

* feat(packs): attach fluency lanes to he_logos_micro_v1

* feat(packs): attach fluency lanes to grc_logos_cognition_v1

* feat(packs): ADR-0103 fluency lane attachment

* test(packs): expect ADR-0103 fluency lanes on Hebrew Greek contracts

* docs(evals): add Hebrew fluency holdout split note

* docs(evals): add Koine Greek fluency holdout split note

* docs(evals): note Hebrew holdout attachment

* docs(evals): note Koine Greek holdout attachment

* docs: add ADR 0103 placeholder

* docs(adr): expand ADR-0103 fluency lane attachment

* docs: index ADR-0103 and refresh frontier
2026-05-22 09:43:46 -07:00
Shay
57fa078aaa docs: mark ADR-0094/0095/0098/0099 accepted
Sibling reconciliation PR to #104. The four ADRs explicitly called out as
the 'current implementation frontier' in PR #104 are already implemented
to the same evidence bar as the eight ADRs that PR accepted:

- ADR-0094: teaching/source.py + proposal schema widening + migration
  script; tests/test_proposal_source.py green
- ADR-0095: teaching/from_miner.py + miner_loop_closure lane;
  SHA-pinned in scripts/verify_lane_shas.py; tests/test_miner_proposals.py
  green
- ADR-0098: core/demos/contract.py + adapter surface + demo_composition
  lane; SHA-pinned; tests/test_demo_composition.py green
- ADR-0099: core/demos/showcase.py + public_demo lane;
  SHA-pinned; tests/test_public_showcase.py green

Three of four lanes are SHA-pinned in CI (a stricter bar than several
already-accepted ADRs). Local pytest run: 85/85 passed across the four
tests/test_*.py files in 17s.

Also refreshes docs/decisions/README.md:
- flips the four table rows to Accepted (2026-05-22)
- rewrites the 'Current frontier' section now that no ADR-0091..0102
  entry is unimplemented
- enumerates candidate next directions (curriculum proposals,
  language-specific holdout splits, expert-demo ratification)

Docs-only change; no runtime code touched.
2026-05-22 07:09:14 -07:00
Shay
fe8d6db224 docs: update ADR index and current frontier 2026-05-22 06:47:25 -07:00
Shay
f21b1a9ba6 docs: mark ADR 0102 accepted 2026-05-22 06:42:27 -07:00
Shay
dde928859c docs: mark ADR 0101 accepted 2026-05-22 06:34:35 -07:00
Shay
2bc3e67474 docs: mark ADR 0100 accepted 2026-05-22 06:30:53 -07:00
Shay
d0d1a96669 docs: mark ADR 0097 accepted 2026-05-22 06:28:51 -07:00
Shay
852585a83d docs: mark ADR 0096 accepted 2026-05-22 06:27:25 -07:00
Shay
32d6ff6576 docs: mark ADR 0093 accepted 2026-05-22 06:25:38 -07:00
Shay
cf8ad47c53 docs: mark ADR 0092 accepted 2026-05-22 06:24:03 -07:00
Shay
87762e34fa docs: mark ADR 0091 accepted 2026-05-22 06:21:59 -07:00
Shay
b9a6f2ddb5 feat(packs): ADR-0100/0101/0102 — three sibling domain ratifications
Ratifies the remaining three sibling domains as reasoning-capable
under ADR-0091's Domain Pack Contract v1, using the template
ADR-0097 established for mathematics_logic. The capability ledger
now has four reasoning-capable rows backed by validated contracts.

ADR-0100 physics (en_physics_v1):
  domain_id: physics
  claimed_operators: causal, modal
  teaching_chains: [physics_chains_v1]
  eval_lanes: foundational_physics_ood, inference_closure,
    fabrication_control
  9/9 predicates pass

ADR-0101 systems_software (en_systems_software_v1):
  domain_id: systems_software
  claimed_operators: transitive, causal
  teaching_chains: [systems_software_chains_v1]
  eval_lanes: symbolic_logic, inference_closure, fabrication_control
  9/9 predicates pass

ADR-0102 hebrew_greek_textual_reasoning (FIRST MULTI-PACK ratification):
  domain_id: hebrew_greek_textual_reasoning
  claimed_operators: causal, contradiction
  teaching_chains: [hebrew_greek_textual_reasoning_chains_v1]
  eval_lanes: inference_closure, fabrication_control
    (universal lanes only — language-specific fluency lanes lack
    holdout splits; a separate ADR adds those when holdouts ship)
  packs: grc_logos_micro_v1, grc_logos_cognition_v1,
    he_logos_micro_v1, he_core_cognition_v1
  all four pack contracts identical (uniformity invariant pinned);
  all four 9/9 predicates pass
  pre-existing gap: hebrew/greek manifests lacked a provenance field
  entirely; ratification fills that uniformly across the four packs

44 new ratification tests in test_adr_0100_0102_sibling_ratifications.py:
- 6 parametrized 9-predicate validation tests (one per pack)
- 21 per-domain ledger status assertions (status, reasoning_capable,
  expert_demo gated, no_open_gaps, provenance points at correct ADR,
  operator_chain_coverage, intent_shapes minimum) — 7 cases × 3 domains
- 15 per-domain contract field shape assertions (teaching_chains,
  eval_lanes, splits coverage, axioms/rules null, primary reviewer) —
  5 cases × 3 domains
- 2 ADR-0102 multi-pack uniformity invariants (all four packs carry
  the contract; contracts identical across packs)

Capability ledger after ratification:
  systems_software           : reasoning-capable
  mathematics_logic          : reasoning-capable
  physics                    : reasoning-capable
  hebrew_greek_textual_reasoning : reasoning-capable
  philosophy_theology        : reasoning-capable (no contract; pre-existing)

Lane SHA pin update:
- public_demo pin refreshed (21751aaf.. → 71090323..) — the
  ratification adds new manifest fields (provenance,
  domain_contract_*) that surface in pack-related demo paths;
  intentional ADR-tracked change per the verifier doctrine

Smoke 67/67, packs 6/6, sibling ratifications 44/44, cognition eval
byte-identical 100/100/100/100; all 6 lanes match pinned SHAs:
  reviewer_registry            681a2aab..
  miner_loop_closure           9f071733..
  domain_contract_validation   f9c06cde..
  fabrication_control_summary  01e1b6b7..
  demo_composition             27d83824..
  public_demo                  71090323..
2026-05-21 20:25:48 -07:00
Shay
afdd2ee413 feat(capability): implement ADR-0092 — Reviewer Registry v1
Closes the load-bearing gap blocking every reasoning-capable claim
under ADR-0091: docs/reviewers.yaml was previously `reviewers: []` and
unparsed. Now schema-validated at v1, with a bootstrap shay-j entry
self-sealed via provenance.

- new core.capability.reviewers module: frozen Reviewer/ReviewerRegistry
  dataclasses, strict load_reviewer_registry parser, ReviewerRegistryError
- enforces ADR-0092 schema rules: schema_version==1, no unknown
  top-level keys, no unknown reviewer fields, role∈{primary,domain},
  primary must claim ["*"], domain must NOT claim "*", review_scope
  subset of {pack,proposal,chain,eval}, no duplicate reviewer_ids
- can_review(reviewer_id, domain_id, scope) helper implements
  ADR-0092 rules 2-4 for downstream use by ADR-0093 validator
- docs/reviewers.yaml updated to v1 schema with shay-j bootstrap
- ledger_report() evidence_counts now exposes structured
  reviewer_registry status (valid, schema_version, reviewer_count,
  reviewer_ids, error) alongside the legacy reviewers_present bool
- new evals/reviewer_registry/ lane: 6 cases (2 positive + 4 negative)
  covering empty-registry, wrong-version, domain-wildcard rejection,
  and unknown-field rejection
- runner emits deterministic JSON report; two runs produce byte-identical
  output (sha256 verified)
- 26 unit tests in tests/test_reviewer_registry.py
- capability ledger test extended to assert new reviewer_registry block
- smoke suite green (67/67); lane passes 6/6

The pre-existing test_flag_report_tracks_default_off_flags failure is
unrelated (discourse_planner flag default) and not introduced here.
2026-05-21 18:01:24 -07:00
Shay
10457bec69 docs(adr): propose ADRs 0092-0099 — scale-up slate through public showcase
Eight load-bearing ADRs closing the loop from contemplation Phase 5 through
a public showcase demo. Each one is small and evidence-bearing; together
they sequence the next arc without duplicating existing substrate.

- 0092 Reviewer Registry v1 — populates docs/reviewers.yaml schema;
  unblocks all reasoning-capable claims under ADR-0091.
- 0093 Domain Pack Contract v1 Implementation — wires ADR-0091's five
  follow-up items (parser, dry-run validator, chain registry, eval lane
  refs, reviewer resolution) so manifest fields actually gate status.
- 0094 Proposal Source Provenance — sealed ProposalSource type widening
  proposal schemas ahead of 0095.
- 0095 Miner-Sourced Teaching Proposals — closes the contemplation
  loop: articulation_quality / contradiction_detection / frontier_compare
  miners emit PackMutationProposal candidates routed through the single
  reviewed teaching path; identity-pack defense at construction, not
  review; replay-equivalence pre-gate.
- 0096 Fabrication-Control Eval Lane — first negative-control measure;
  three case classes (phantom endpoint, cross-pack non-bridge, sibling
  collapse) with frozen thresholds (fabrication_rate ≤ 0.01).
- 0097 Mathematics-Logic Reasoning-Capable Ratification — first
  domain claim under ADR-0091; chain corpus + eval lanes already
  exist, this is the formal contract ratification.
- 0098 Demo Composition Contract — DemoCommand protocol so demos can
  be composed without reimplementation; deterministic JSON, no global
  state mutation, declared output paths only.
- 0099 Public Showcase Demo — composes four scenes (determinism /
  honest unknown / reviewed learning / multi-hop+trace) under 30s;
  pure composition enforced by grep gate; JSON byte-equality CI-pinned.

Landing order: 0092 → 0094 → 0095 → 0093 → 0096 → 0097 → 0098 → 0099.

Deliberately not included: curriculum compiler, formation course
runner, calculator operators, response-mode taxonomy expansion,
learning-scale 10k harness. Each is deferred with a documented reason.
2026-05-21 17:34:09 -07:00
Shay
8893962245 docs(sessions): extend articulation-arc note for Phase 5
Phase 5 landed in commit 327047c (articulation-quality miner +
runtime sink wiring + full end-to-end loop tests).  Extending the
session note in-place per its own append-only convention so the
single document covers the complete arc Phases 1–5.

Sections updated
----------------

* §0 executive summary
    - commit count: 9 → 11
    - phases shipped: 4 → 5 (new Phase 5 row in the deliverables
      table)
    - observation surfaces table grows two rows for
      ``attach_articulation_sink`` and
      ``mine_articulation_observations``
    - test artifacts: 6 → 8 files, 64 → 82 cases

* §3.5 NEW — full architectural walkthrough of Phase 5
    - Why the loop closes (links the user's
      "memory confidence scoring" intuition to ADR-0080's
      doctrine-aligned realisation: reviewable evidence, not
      autonomous mutation)
    - File-by-file delta (chat/articulation_telemetry.py + miner
      + runtime wiring)
    - Three v1 mining rules (recurring_predicate_monotony /
      recurring_planner_gap / low_average_predicate_diversity)
    - Loop diagram showing live + offline halves
    - Recorded demo output from the commit message
    - Doctrine pin table mapping each constraint to its test

* §4 pipeline diagram extended to show Phase 5 sink + offline
  miner branches

* §5 verification table gains four new rows for Phase 5 claims
  (full-loop emission, byte-equal finding IDs across two e2e
  runs, JSONL round-trip identity, opt-in gating)

* §5.6 suite totals updated:
    - Contemplation subsuite: 35/35 → 53/53 (Phase 3+4+5)
    - New row for Phase 5 articulation-quality e2e (7/7)

* §6 case study — added the "After Phase 5" trace and the
  closing one-line story across all five phases for one prompt
  ("What is truth, and why does it matter?")

* §7 architecture surfaces table grows a row for
  chat/articulation_telemetry.py and adds the miner to the
  contemplation subsystem row

* §9 inverted from "what would close the loop" (future work) to
  "SHIPPED — here's what it now unlocks":
    - production sink + retention policy
    - additional aggregation rules
    - CLI hook
    - review-loop wiring back into PackMutationProposal

* §10 reference index grows new lines for
  chat/articulation_telemetry.py,
  core/contemplation/miners/articulation_quality.py,
  tests/test_articulation_quality_miner.py, and
  tests/test_articulation_quality_e2e.py

* Footer updated: notes the arc is complete; future arcs should
  start a new session-notes file and cross-link rather than
  rewriting this one.

The session-notes file is now 1100+ lines — the complete frozen
reference for the articulation arc that took CORE from one-sentence
pack-grounded surfaces to a full live-reasoning + offline-mining
+ reviewable-proposals feedback loop, all doctrine-aligned, all in
one session.
2026-05-21 11:03:01 -07:00
Shay
1740b7d518 docs(sessions): articulation arc — comprehensive 2026-05-21 session note
Thorough why/when/where/how reference for the four-phase articulation
arc shipped this session plus the pre-arc classifier/Rust cleanup
that made it possible.  Designed as the load-bearing entry point for
future contributors, case studies, capability audits, and
architectural reviews.

Sections
--------

0.  Executive summary — what was achieved (9 commits, 64 new tests,
    user-visible before/after, doctrine evidence)
1.  Why this work happened (visible gap, doctrine constraint, what
    was already wired vs. missing)
2.  Pre-arc cleanup — RECALL trigger / CORRECTION x2 / Rust FFI
3.  Phase 1 — discourse planner default ON + fast-path
    Phase 2 — reflective rendering (subject pronominalization)
    Phase 3 — live plan contemplation pre-flight
    Phase 4 — per-plan articulation telemetry metrics
4.  The pipeline today (diagram + before/after table)
5.  Verification — every claim and the test that holds it.
    Five sub-tables: doctrine claims, quality claims, back-compat /
    null-lift claims, ADR-0072 structural invariants,
    `core bench --suite all` performance, suite-level totals.
6.  Case study — the compound prompt as a story across all four
    phases
7.  Architecture surfaces touched (which file got what change in
    which phase)
8.  What was deliberately NOT built (and why — connective rotation,
    generalised pronoun selection, plan revision, Phase 2.5,
    Rust algorithmic optimisation)
9.  Phase 5 — what would close the user-intuited "live reasoning
    → memory confidence" loop, doctrine-aligned
10. Reference index — modules, flags, tests, ADR cross-refs

Per the user's request: captures the why/when/where/how thoroughly
so this work is recoverable for future reference, case studies, and
building on top.  Append-only convention: future sessions extending
this arc should add new sections below rather than rewriting — the
history is itself evidence of the doctrine working in practice.
2026-05-21 10:45:26 -07:00
Shay
07b44b0920
docs(handoff): two Codex briefs — domain-pack closure cleanup + grc/he content phase II (#98)
Brief 1 scopes the PR #97 follow-up: flip `definitional_layer` to false on
the three new domain seed packs (en_mathematics_logic_v1 / en_physics_v1 /
en_systems_software_v1) so the ADR-0084 closure verifier stops flagging
them as malformed. They're governed by ADR-0091 `domain_contract_version`,
which is a different layer; the integration test's allowlist already
excludes them, so the flag-flip aligns the script with the test's
clearly-intended scope.

Brief 2 scopes ADR-0073 L1.1 content phase II: extend the grc/he anchor-lens
substrate with 8 cross-language lemma pairs (νοῦς ↔ בינה / διάνοια /
καρδία ↔ לב / ψυχή ↔ נפש / ἔλεος / εἰρήνη / δικαιοσύνη / ἅγιος ↔ קדוש)
authored against the L1.1 file shapes, alignment edge weights, and
en-collapse annotation discipline. Anchor-lens-tour seam claims and
cognition-eval byte-identity under unanchored default remain the gates.

Both briefs are bounded to content authoring; neither touches code.
2026-05-20 23:12:15 -07:00
Shay
3d922a1532
Add chain-first capability ledger and domain seeds (#97) 2026-05-20 21:33:24 -07:00
Shay
a36b48b198
feat(runtime): opt-in unified-ingest path (ADR-0090, audit Findings 6+7) (#95)
Closes audit Findings 6 (within-turn recall not batched) and 7
(probe-ingest / commit-ingest dual field) as a single PR — the two
are architecturally entangled and resolve together.

Pre-fix flow in ``ChatRuntime.chat()``:

  1. ``probe_ingest(filtered)`` → ``probe_state.F``
  2. Gate check on ``probe_state.F``
  3. If gate fires: ``commit_ingest`` + stub response
  4. Otherwise: ``commit_ingest`` + drive bias → ``field_state.F``
  5. Walk runs on ``field_state.F``

The gate observes one manifold position; the walk navigates a
slightly different one (drive bias applied between them).  Honest
refusal decisions and walk outputs are made on different fields —
the audit's named coherence gap.

This PR ships a flag-gated unified-ingest path following the
codebase's standard substantive-change pattern (ADR-0046 /
ADR-0062 / ADR-0085 / ADR-0088 / ADR-0089):

``RuntimeConfig.unified_ingest: bool = False`` (default).

When ``True``:

  1. ``commit_ingest(filtered)`` runs first.
  2. Drive bias applied immediately.
  3. Gate observes ``committed.F``.
  4. If gate fires: stub response (turn has already committed —
     intentional semantic change documented in ADR-0090).
  5. Otherwise: walk runs on the same ``committed.F`` the gate
     decided against — no second ``commit_ingest`` call.
  6. ``probe_ingest`` is not called on this path.

When ``False`` (default): historical behavior is preserved
bit-for-bit; ``probe_ingest`` still runs first.

ADR-0090 documents:

  * Phase 1 (this PR): unified-ingest substrate.
  * Phase 2 (separate PR, after Phase 1 validates): batched recall
    — pass the gate's ``direct_hits`` into ``generate()`` as a
    ``prebuilt_first_recall`` so the walk's first step does not
    re-call ``vault.recall()`` on the same field.  Single recall
    call eliminated per turn.
  * Out of scope: ``recall_batch`` for per-step walk recalls
    (each step's query depends on the previous step's field
    state; not batchable without changing walk geometry).

Validation:

  * 5 new tests in ``tests/test_unified_ingest_null_lift.py``:
      - flag defaults to ``False`` on ``DEFAULT_CONFIG``
      - flag-off surface + trace_hash + vault_hits byte-identical
      - flag-on does not call ``probe_ingest`` (verified via spy)
      - flag-on produces well-formed surface + trace_hash
      - flag-off still calls ``probe_ingest`` (historical guard)
  * ``core eval cognition`` byte-identical across all three splits:
    public 100/100/91.7/100, dev 100/100/78.6/100, holdout
    100/100/83.3/100.
  * ``core test --suite cognition`` 120/0/1, ``smoke`` 67/0,
    ``runtime`` 19/0.

Comb-pass status after this PR:

  * Item 4 (graph topo) ✓ #92
  * Item 5 (realizer node_map) ✓ #91
  * Item 6 (batch recall) ✓ ADR-0090 substrate (this PR); Phase 2
    optimization is queued
  * Item 7 (probe/commit dual ingest) ✓ ADR-0090 (this PR)
  * Item 8 (dead defensiveness sweep) ✓ #91
  * Item 9 (local imports) ✓ #91
  * Item 11 (dead ``_fold_compose_into_surface``) ✓ #91
  * Item 13 (``_serialize_*`` fold) ✓ #91
  * Item 15 (GenerationResult tuple/list) ⊘ false positive
  * Item 16 (subject normalization consistency) ✓ #93
  * Item 17 (redundant ``^`` anchors) ✓ #94
  * Tier 5 minor (``_BE_FORMS`` hoist, walrus, reverse-iter) ✓ #94
2026-05-20 21:00:27 -07:00
Shay
5446ab1615
docs: pack inventory 2026-05-21 — strengths, weaknesses, leverage gaps (#84)
Comprehensive survey of every pack in the tree across five layers
(primitives → language → teaching → policy → selection axes → style)
with per-pack stats, cross-pack lemma overlap, teaching-chain graph
topology, and a ranked top-leverage-gaps list.

Key findings:

  * **Layer 4 (selection axes) is the densest by far** — 24 packs
    (17 anchor lenses + 7 registers).  Greek and Hebrew lens
    families are at parity (8 each).
  * **Layer 2 (teaching corpora) is the load-bearing thin layer** —
    only 41 reviewed chains across 18 subjects, with just 2 intent
    shapes covered (CAUSE / VERIFICATION) and 7 connectives total.
    No COMPARISON / PROCEDURE / CORRECTION chains exist; those
    intents route through pack composers only.
  * **Layer 1 EN is solid** — 354 lemmas across 12 mounted packs
    with 93% gloss coverage; 24 ungloss'd lemmas remain.
  * **Cross-language is structurally present but content-light** —
    grc / he micro-packs ship and mount; the bigger
    ``*_cognition_v1`` siblings ratify but are not in the default
    mount; no glosses exist on any non-EN pack.
  * **Three drafted ethics packs are unratified** (legal /
    research / engineering) — the highest-leverage low-effort gap.
  * **Rhetorical-style axis ships substrate only** — one
    null-lift pack; ADR-0087 consumer phase pending.

Top-leverage gaps ordered by ratio of unblocked value to effort
are enumerated in §7 of the doc itself.  No code lands here.
2026-05-20 19:59:18 -07:00
Shay
8b51813c73
docs(adr): ADR-0088 + ADR-0089 — unblock Findings 2 + 4 (#81)
Two ADRs that unblock the remaining items from the 2026-05-20 audit
that could not ship as direct PRs.

ADR-0088 — Realizer-Grounded Authority (Finding 2 retry)
========================================================

The first-response audit remedy (wire ``ground_graph`` between
``runtime.chat`` and ``realize_semantic``) was empirically attempted
on ``fix/ground-graph-wiring`` and reverted: the grounded realizer's
template output (e.g. ``"Light is a visible medium that reveal
truth"``) is grammatically and stylistically weaker than the runtime
path's ADR-0085-polished pack-grounded surface, so the realizer wins
the surface resolver (PR #76) and the user-visible surface regresses
on 23 byte-identical tests + ``register_invariant_grounding``.

ADR-0088 reframes Finding 2 as a two-phase rollout:

  * Phase A (no behavior change) — realizer fluency parity.
    Templates consult the same gloss source ADR-0085 wired into the
    CAUSE composer, emit 3sg verb agreement, and carry the same
    pack-provenance tag the runtime path emits.  Byte-identical
    today because the realizer is still gated by
    ``_is_useful_surface``.
  * Phase B (substantive) — ground the graph and let the realizer
    compete.  Surfaces change exactly once, with a per-case
    re-baseline justified by a "fluency ≥ pre-fix runtime surface"
    invariant.

The audit's final-draft remedy (hot-path short-circuit only) is
explicitly rejected — pure perf cleanup, no metric lift since
``core eval cognition`` is already at 100% groundedness.

ADR-0089 — Compound-Intent Pipeline Dispatch (Finding 4)
========================================================

``classify_compound_intent`` is implemented but never reaches
``CognitiveTurnPipeline.run()``.  Compound inputs like *"What is X
and how does it relate to Y?"* silently drop the second clause.

Naive multi-node dispatch breaks every downstream stage:
PropositionGraph (one root), plan_articulation (single-root),
realize_semantic (one target), surface resolver (one surface per
turn), compute_trace_hash (one intent_tag + articulation_surface),
teaching loop (one correction-source proposal), register / anchor-
lens telemetry (one variant per turn).

ADR-0089 proposes a three-phase rollout:

  * Phase C1 (no behavior change) — call ``classify_compound_intent``
    in step 1b, record dropped clauses on
    ``CognitiveTurnResult.dropped_compound_clauses`` for
    observability while routing the dominant clause through the
    existing single-intent path.
  * Phase C2 (opt-in substantive) — flag-gated multi-node graph
    dispatch with new ``CompoundEdge`` / ``ConjunctionRelation``,
    widened ``compute_trace_hash`` carrying a
    ``compound_clauses_hash``, and a ``multi_clause_surface`` field
    on the resolver.  Flag-off preserves byte-identity.
  * Phase C3 — telemetry alignment + demo + docs.

Each phase is independently shippable and preserves the existing
null-lift / byte-identity invariants register and anchor-lens
established (ADR-0072, ADR-0073d) as the project's pattern for
substantive runtime behavior changes.

Both ADRs are Proposed; ratification follows the existing pack /
ADR review process.  No code lands in this commit.
2026-05-20 19:59:14 -07:00
Shay
886fc4553f
docs(adr-0085): content style pass v2 brief — 3sg + plural agreement (#75)
Follow-up brief for the cheaper dev agent.  Scoped tight: ~18 concrete
row-by-row edits across two patterns the v1 pass (PR #73) deferred or
missed.

Pattern A — 3sg present-tense agreement after relative pronouns
  what/who/that/which followed by a bare-form verb where the implied
  subject is singular.  10 candidates identified up front by repo scan:
    - en_core_causation_v1/effect       (with judgment note)
    - en_core_cognition_v1/beginning, creation, definition, evidence,
      light, reason, symbol
    - en_core_meta_v1/example, mind

  Brief explicitly clarifies that bare verbs after modals (`can`,
  `may`, `should`) are CORRECT and must NOT be changed — flagging
  the "who can know and do" case the v1 agent did right.

Pattern B — plural agreement after quantifier / preposition
  between/among/of/two/three/many + count-noun-in-singular.  5
  confident edits + 3 borderline cases with judgment guidance:
    - en_core_attitude_v1/broad
    - en_core_cognition_v1/context, order, style
    - en_core_spatial_v1/between
  Borderline guidance distinguishes count vs mass nouns: `reason`
  in "group of reason" is count (apply fix); `reason` in "because
  of reason" is mass (leave alone).

Same hard rules as v1 brief:
  - no code edits
  - definitional_atoms / predicates_invited / pos / lemma /
    definition_version must not change
  - Greek/Hebrew packs and primitives pack untouched
  - closure verifier must exit 0
  - cognition eval must stay byte-identical
  - draft PR, human review before merge

Estimated effort: ~18 one-character edits.  Whole pass should take an
order of magnitude less time than v1 because the candidate rows are
enumerated in the brief itself rather than discovered via heuristic
scan.

Why a v2 brief rather than amending v1's PR:
  Plural patterns were not on v1's explicit pattern list (the v1
  brief only named verb agreement, missing articles, missing
  infinitives, missing copulas).  3sg agreement was named but
  required a different tool than the agent had at hand.  Scoping
  v2 to exactly the rows known to need fixing is cheaper than
  re-running the v1 heuristic scan with an expanded ruleset.

Provenance tag for the v2 pass:
  adr-0085-style-v2:reviewed:2026-05-22
2026-05-20 16:23:14 -07:00
Shay
ac75cfc659
docs(adr-0087): rhetorical style as selection axis + writing-chain harvester spec (#72)
Pre-work for a writing-curriculum extension to CORE.  Two companion
documents, both Proposed status (no code shipped).

docs/decisions/ADR-0087-rhetorical-style-axis.md
  Pins rhetorical style as a third selection axis — sibling to anchor
  lens (ADR-0073), orthogonal to register (ADR-0070).  Substantive
  axis: trace_hash DISTINCT across styles (style changes which moves
  the composer requires and which frames the realizer emits, which
  changes the propositional plan, which changes the trace).

  Four anti-patterns explicitly named and rejected:
    - style as motor (re-couples realizer to geometry; same shape as
      the ADR-0085 fusion-operator rejection)
    - style as register variant (conflates substantive with stylistic)
    - style as identity axis (bloats identity doctrine)
    - style auto-detected from user input (operator-chosen only)

  Pack shape mirrors packs/anchor_lens/.  default_unstyled_v1 is the
  null-lift pack identical to no-style behavior.  Three CI invariants
  proposed: rhetorical_style_null_lift, schema validation, three-axis
  orthogonality.

  Substrate-only ADR — no consumer code, no genre packs.  Consumer
  integration is a follow-up ADR (composer + realizer extensions
  that read permitted_frames + required_moves_per_claim +
  forbidden_moves).

docs/curriculum/writing-chain-harvester-spec.md
  Layer 0 of the writing curriculum.  A deterministic tool that
  extracts candidate (subject, predicate, object) triples from
  reviewed expert prose and surfaces them as proposals to the
  existing teaching/review pipeline.

  Five stages (segment → classify → extract → propose → audit) —
  pure-Python rule-based, no LLM generation, no auto-acceptance.
  Trust boundary: reviewer accept/reject via the existing
  core teaching propose/review path.  No bypass permitted.

  The harvester is a proposal PRODUCER, not a proposal CONSUMER.
  Plugs into the existing pipeline without inventing a new review
  mechanism.  Each proposal carries source_id + source_line + the
  exact source_clause it came from for reviewer verification.

  First-implementation acceptance criteria deliberately tight:
  Stage 0+1 with dry-run only.  Stages 2-5 are follow-up PRs.

Substrate-first sequencing pattern (ADR-0084 → 0085) reused
throughout.  Both documents acknowledge open questions deferred to
implementation phase rather than pre-deciding.

Why now: a writing curriculum is being scoped.  Without this ADR,
every downstream PR faces the same "should style be a motor?"
question and the temptation to reach for the geometry will recur
every time the realizer produces a stilted surface.  Pinning the
axis up front prevents that recurrence.
2026-05-20 16:09:16 -07:00
Shay
537d73e394
docs(adr-0085): content style pass brief for the cheaper dev agent (#71)
Brief for a fluency pass on the 333 ratified gloss entries. Closes the
content-side counterpart of ADR-0085's surface lift:

  Before  "Light exists as visible medium that reveal truth."
  After   "Light exists as a visible medium that reveals truth."

Same gloss content, English-correct. Fixes 3sg agreement after
relative clauses, plural agreement after quantifiers, missing
articles before adjective-noun NOUN-frame glosses, and missing 'to'
in VERB-frame glosses.

Hard constraints encoded in the brief (matching the wrapper-prompt
pattern that worked for ADR-0084 content):

- code untouched: only language_packs/data/<pack>/glosses.jsonl edits
- definitional_atoms, predicates_invited, pos, lemma, definition_version
  must NOT change
- closure verifier (scripts/verify_definitional_closure.py) must
  still exit 0
- cognition eval must stay byte-identical to baseline
- Greek/Hebrew packs untouched (not in definitional layer per ADR-0084
  scope limit)
- primitives pack untouched
- draft PR, human review before merge

Includes a Phase-1 inventory script the agent runs first to scope the
work (heuristic pattern matcher across the 13 opted-in packs),
worked-example fluency rules with before/after table, per-pack
checksum-refresh shell snippet, and Phase-4 verification commands.

Estimated effort: ~30-60 lines of JSONL edits. Same handoff format as
docs/handoff/ADR-0084-pack-content-brief.md.
2026-05-20 15:58:44 -07:00
Shay
4b9404a88e
feat(adr-0085): gloss-aware CAUSE composer — explanation frame from glosses (#70)
The original "Why does light exist?" complaint that motivated ADR-0084
was specifically about CAUSE-intent surfaces. ADR-0084 (substrate) +
PR #65 (content) already moved DEFINITION/RECALL to gloss-grounded
surfaces ("Light is visible medium that reveal truth."). But CAUSE
still dispatched through the chain-walk path:

  Before: light — teaching-grounded (cognition_chains_v1):
            cognition.illumination; logos.core.
            light reveals truth (cognition.truth).
            No session evidence yet.

  After:  Light exists as visible medium that reveal truth.
          pack-grounded (en_core_cognition_v1).

The chain-walk is structurally correct but the wrong SHAPE for a why-
question — it's a graph traversal, not an explanation. ADR-0085 fixes
the shape using the same gloss material that DEFINITION/RECALL already
consume, with no new content authoring.

Additive composer
  chat/pack_grounding.py:gloss_aware_cause_surface()
  - Resolves gloss via lexicon-residency-checked resolve_gloss().
  - Frames POS-aware:
      NOUN -> "{Lemma} exists as {gloss}."
      VERB -> "To {lemma} is to {gloss}."
      ADJ  -> "To be {lemma} is to {gloss}."
      *    -> falls back to _frame_gloss (predicate-identity).
  - Threads anchor lens via the existing helper (ADR-0073c parity).
  - Returns None when no gloss exists — runtime falls through to the
    existing chain-walk path. Additive: no CAUSE case loses its surface.

Runtime dispatch
  chat/runtime.py — IntentTag.CAUSE tries gloss path FIRST under the
  flag; falls through to teaching_grounded_surface* on None.
  Unconditional fallback — never silent.

Opt-in flag
  core/config.py — RuntimeConfig.gloss_aware_cause: bool = False
  Default off preserves pre-ADR-0085 chain-walk surfaces byte-
  identically (null-drop invariant, CI-pinned).

Prompt-diversity classifier update
  evals/prompt_diversity/runner.py — _CAUSE_MARKERS widened with the
  explanation-frame markers ("exists as", "is to", "to be", "is for",
  "purpose of") plus bare-form predicates ("reveal" alongside
  "reveals"). Neither composer path is penalised on shape_fit just on
  inflection grounds.

v1/public lift (flag OFF vs ON, 26 cases)
  intent_accuracy        : 65.4% -> 65.4%   ( — )
  versor_closure_rate    : 100.0% -> 100.0% ( — )
  response_shape_fit     : 57.7% -> 57.7%   ( — , both frames recognized)
  audit_in_surface_rate  : 42.3% -> 42.3%   ( — , envelope ADR's job)
  gloss_quote_rate       : 11.5% -> 23.1%   (+11.5pp, structural lift)

Tests (15)
  - 5 pure composer (NOUN/VERB frame, unknown/empty None, no chain-
    walk artifacts in surface)
  - 5 runtime dispatch (flag-off chain-walk, flag-on gloss, parametrized
    across glossed subjects, VERIFICATION unchanged under flag, no-
    gloss fallback engages)
  - 5 cognition lane invariance (aggregate metrics byte-identical
    under both flag states; surfaces deliberately shift on the 2 CAUSE
    cases with glossed subjects — the structural-change-vs-metric-
    invariance both-sides invariant)

Lanes
  smoke 67/0, cognition 120/0/1 skipped, packs 6/0, teaching 17/0,
  runtime 19/0. core eval cognition byte-identical 100/91.7/100/100
  under both flag states.

Scope limits (per ADR §Scope limits)
  - CAUSE only; VERIFICATION still chain-walks (different shape).
  - English pilot only; Greek/Hebrew packs not opted into definitional
    layer yet (ADR-0084 scope limit).
  - Single-lemma subjects; compound/anaphoric fall through.
  - Opt-in until cognition holdout confirms the lift transfers off-
    fixture. Future PR flips default on.

Out of scope
  - Surface-vs-envelope cleanup ("pack-grounded (...)" still leaks).
  - Predicate licensing (ADR-0086).
  - Content style pass (bare lemma forms in glosses — separate brief).
2026-05-20 15:55:08 -07:00
Shay
48282eef8d
feat(adr-0084): definitional layer — proposal + substrate (schema/loader/closure) (#64)
* docs(adr-0084): propose definitional layer + prompt-diversity suite

Three companion artifacts proposing the next substantive design step
after ADR-0083:

1. ADR-0084 (Proposed) — Definitional Layer for Lexicon Packs
   Optional `definition` block on pack entries: gloss,
   definitional_atoms, predicates_invited, definition_version,
   provenance.  Pack-level opt-in.  Closure rule: every word in a
   gloss must resolve to a same-pack lemma, another mounted pack's
   lemma, or a primitive in a new `packs/primitives/` pack.
   NO composer change in this ADR (sequenced for ADR-0085) —
   ratify substrate before any consumer depends on it.

2. evals/prompt_diversity/ (Proposed) — companion eval lane
   ~50 cases across question-shape × sophistication × domain,
   measuring three new metrics: response_shape_fit,
   audit_in_surface_rate (quantifies the trust-boundary leak into
   user surfaces), gloss_quote_rate (zero today; rises with future
   gloss-aware composer).  No v1 pass thresholds — the lane
   establishes a baseline distribution so future work has
   something to move.  26 seed cases authored covering all 21
   categories.

3. docs/handoff/ADR-0084-pack-content-brief.md — paste-ready brief
   for a cheaper/faster dev agent to produce the pack content in
   parallel.  Self-contained, 5 sequenced phases (primitives pack
   → extend 9 existing glosses → add to relations/anchors → write
   closure verifier → run safety lanes), explicit don't-touch list
   (no composer / runtime / algebra / Greek+Hebrew packs / schema
   parser), no-LLM-glosses discipline, per-phase acceptance.

Discovery while drafting: 9 packs already carry glosses.jsonl
under language_packs/data/ with a flat schema (78 entries in
en_core_cognition_v1 alone).  The brief reflects that — most
work is extending existing entries, not authoring from scratch.

Strategic context: ADR-0083 raised the *depth* ceiling on chain
composition; ADR-0084 raises the *fidelity* ceiling.  The φ
separation probe (memory: phi-separation-falsified) established
that semantic capability lives in chain composition, not in φ
geometry, so deepening the composer's substrate is the natural
next step.  ADR-0084 → 0085 (gloss-aware composer) → 0086
(predicate licensing at ratification) is the planned sequence.

* feat(adr-0084): substrate — schema parser, primitives loader, closure verifier

Substrate-only code-side for ADR-0084 (Definitional Layer for Lexicon Packs).
No composer touches the new fields yet; consumer integration is ADR-0085.

Schema (additive, default preserves byte-identity)
  - LanguagePackManifest.definitional_layer: bool = False
  - compiler loader propagates the flag from manifest.json

language_packs/definitions.py (new)
  - GlossEntry dataclass: lemma, gloss, pos, definitional_atoms,
    predicates_invited, definition_version, provenance_ids
  - parse_gloss_entry(payload, *, strict) — strict mode enforces ADR-0084
    §Schema validation row-by-row: required keys, typed lists, no
    unknown keys, positive definition_version; lax mode preserves the
    legacy two-field shape for back-compat
  - load_pack_glosses(pack_id, *, strict) with cache + clear hook
  - verify_definitional_closure(pack_id, *, mounted_pack_lemmas,
    primitive_lemmas, strict) returning tuple[ClosureViolation, ...];
    case-insensitive resolution; cycles permitted per ADR

packs/primitives/loader.py (new)
  - Sister loader to packs/safety/ and packs/identity/
  - PrimitivesPack frozen dataclass with .lemmas frozenset
  - Gates: checksum match, kind=='primitives', definitional_layer:true,
    never_auto_mutable:true, pack_id matches dir, primitive_count
    cross-check, duplicate-lemma rejection, path-traversal rejection,
    strict per-entry schema with allow-list
  - DEFAULT_PRIMITIVES_PACK = 'en_semantic_primitives_v1'

tests/test_adr_0084_definitional_substrate.py
  - 38 tests covering strict parser (each required key rejection, unknown
    key rejection, empty predicates_invited allowed, empty
    definitional_atoms rejected, invalid definition_version), lax
    parser back-compat, load_pack_glosses (missing/strict raise/lax
    skip/malformed JSON), closure verifier (same-pack/primitive/mounted/
    unresolved/case-insensitive), primitives loader (every gate), and
    a back-compat check that every shipped pack still ratifies with
    definitional_layer=False

Lanes: smoke 67/0, cognition 120/0/1, teaching 17/0, runtime 19/0,
packs 6/0. Cognition eval byte-identical 100/91.7/100/100.

When the content PR lands (primitives.jsonl + extended glosses.jsonl
under ADR-0084-pack-content-brief.md), the gate catches any closure-rule
violation without further code change.

* feat(evals): prompt_diversity lane runner — measurement instrument for ADR-0084+

Implements the runner against the existing contract.md + 26-case v1
public split.  Lane auto-discovered by evals.framework via the standard
contract + runner convention.

Runner (evals/prompt_diversity/runner.py)
  - run_lane(cases, *, config, workers) -> LaneReport
  - 5 metrics: intent_accuracy, versor_closure_rate (carried over from
    cognition), plus the three new lane-specific metrics —
    response_shape_fit, audit_in_surface_rate, gloss_quote_rate
  - breakdown dict groups by (question_shape, sophistication, domain)
    per contract §How to read the output
  - mirrors evals.cognition.runner's parallel worker pattern

Per-shape classifier (deliberately substring/regex-simple at v1)
  - predicate_identity, explanation, sequence, two_subject_contrast,
    narrative, honest_disclosure
  - Unknown shape => neutral pass (don't penalise new categories)

Audit-leak detector
  - trust-boundary preamble markers (teaching-grounded (, pack-grounded
    (, No session evidence yet.)
  - dotted semantic-domain tag regex (cognition.illumination, etc.)

Gloss-quote detector
  - resolves expected_terms via chat.pack_resolver.resolve_gloss
  - 4-token contiguous-window match against surface (high-confidence
    "gloss actually quoted", not "shared one common word")

Tests (tests/test_prompt_diversity_runner.py — 23)
  - shape classifier parametrized over the six expected_shape values
  - audit-leak detector parametrized over preamble + tag + clean cases
  - end-to-end on v1 public:
      * versor_closure_rate == 1.0 (only v1 pass threshold per contract)
      * every metric in [0, 1]
      * breakdown groups present with the four per-cell metrics
      * diversity gate: >=5 question shapes, >=3 domains
        (defends against future regressions that collapse the suite
         back to a cognition-shaped fixture)

v1/public baseline (26 cases)
  intent_accuracy      : 65.4%   (contract predicted 70-85%)
  versor_closure_rate  : 100.0%  (only v1 pass threshold)  PASS
  response_shape_fit   : 53.8%   (contract predicted low)
  audit_in_surface_rate: 42.3%   (contract predicted ~100%)
  gloss_quote_rate     :  7.7%   (contract predicted 0%)

Three baseline surprises worth noting in the report (NOT failures —
the v1 lane is explicitly there to establish the distribution):

  - audit_in_surface_rate at 42% (not 100%) means the chain-walk leak
    fires on ~11/26; the other 15 are honest-disclosure cases that
    emit no audit envelope.  Sharpens the future surface-vs-envelope
    ADR's actual target: grounded surfaces specifically.
  - response_shape_fit at 54% (not "low") — classifier likely has
    false positives on the ", which " cause-marker.  Worth tightening
    once we have an ADR-0085 baseline to compare against.
  - intent_accuracy at 65% (below predicted 70-85%) — classifier dips
    harder on adversarial/cross-pack than expected.  Real gap.

All five smoke/cognition/teaching/runtime/packs lanes still green;
core eval cognition byte-identical 100/91.7/100/100.

* feat(packs): ADR-0084 pack content (primitives + extend glosses + closure verifier) (#65)

* feat(packs): ADR-0084 pack content

* feat(packs): repair ADR-0084 definitional content

* test(adr-0084): adjust substrate manifest tests for post-#65 content reality

PR #65 flipped definitional_layer:true on 13 English packs (9 core +
4 relations + collapse-anchors).  The substrate's previous test
test_existing_packs_unchanged asserted that en_core_cognition_v1 +
en_core_relations_v1 still had definitional_layer:False — which was
the right pre-content invariant but is wrong post-content.

Replace it with two complementary tests that hold against real content:

  - test_non_opted_packs_default_false:
      pins that packs that DIDN'T flip the flag (en_minimal_v1,
      he_core_cognition_v1, grc_logos_cognition_v1) still surface
      definitional_layer=False through the loader.  Defends against
      a future change accidentally flipping the flag on a non-opted
      pack.

  - test_opted_packs_carry_flag:
      pins that packs that DID flip the flag (en_core_cognition_v1,
      en_core_relations_v1) surface definitional_layer=True through
      the loader.  Proves the substrate's manifest-field propagation
      works against real ratified content, not just fixture packs.

Net: +1 test, same intent (substrate ratifies the manifest field
correctly), now with real-content coverage on both sides of the gate.

All 62 ADR-0084 substrate + prompt-diversity tests pass.
2026-05-20 15:25:25 -07:00
Shay
9e6fa4be75
feat(adr-0083): transitive (multi-hop) teaching-grounded surface (#63)
Strict superset of ADR-0062's depth-1 composer.  `max_depth` is the
number of follow-up hops appended beyond the initial chain:

  max_depth=0  → byte-identical to single-chain surface
  max_depth=1  → byte-identical to ADR-0062 composed
  max_depth=2  → byte-identical to ADR-0062 when no second hop
                 survives, strict superset when one does

The composer surfaces content the realizer was silently dropping
from chains already ratified in `cognition_chains_v1`.  Example
live lift on `"Why does light exist?"`:

  composed: "light reveals truth, which grounds knowledge."
  transitive(2): "...which grounds knowledge, which requires evidence."

Cycle-safe at every depth via a single visited-set; single-corpus
traversal in v1 (cross-corpus transitive deferred to a follow-up
ADR alongside ADR-0064's cross-pack model).

Both flags default False — every existing surface is preserved
byte-identically.  When both `composed_surface` and
`transitive_surface` are True, transitive wins.

Implementation:
- `core/config.py`: `transitive_surface: bool = False`,
  `transitive_max_depth: int = 2`.
- `chat/teaching_grounding.py`: `_resolve_followup` shared helper
  refactored out of the depth-1 composer (no behavioural change),
  plus new `teaching_grounded_surface_transitive(subject,
  intent_tag, *, max_depth)`.
- `chat/runtime.py`: dispatch order — transitive > composed > single.

Verification:
- tests/test_transitive_surface.py: 16 new tests covering pure-fn
  contract, visited-set cycle guard at every depth, runtime
  integration, and the cognition-lane null-drop invariant at
  `max_depth=2` (public + holdout splits).
- tests/test_composed_surface.py: 11/11 pass after the helper
  refactor (ADR-0062 behaviour preserved).
- `core test --suite smoke`: 67 pass.
- `core test --suite cognition`: 120 pass, 1 skipped.
- `core test --suite teaching`: 17 pass.
- `core eval cognition`: 100 / 91.7 / 100 / 100 (byte-identical).
2026-05-20 14:11:40 -07:00
Shay
8f1903e8e7
chore(evals): contracts + bench json + Lane B viewer + chart + audit + demo schema (#62)
* chore(evals, cli): contract standardization + bench --json stdout cleanliness

End-of-session shippability pass.  Three concrete fixes:

1. core/cli.py — bench --json no longer pollutes stdout
   Several bench paths call scripts.run_pulse.run_pulse which prints
   verbose [pulse] traces unconditionally to stdout, breaking jq /
   programmatic consumers of --json output.

   New _bench_stdout_guard() redirects stdout → stderr for the
   duration of the bench run when --json is set.  Operator still sees
   the pulse trace (on stderr), but --json consumers get a clean JSON
   document on stdout.  Applied to all four bench paths: cost,
   articulation, default suite, and --suite all.

   Verified: core bench --suite determinism --json now produces
   parseable JSON; human path still shows 1140 [pulse] lines.

2. evals/{frontier_compare,realizer_guard}/contract.md (new)
   core/contemplation/contract.md (new)

   Each new contract follows the established pattern (37 contracts
   already exist under evals/<lane>/contract.md):

     - What it measures
     - Why it matters (structural win)
     - How to run
     - How to read the output
     - Pass criteria table
     - When it has failed and why
     - Runner / module layout

   Coverage:
     - frontier_compare: both Lane A (CORE-only suites) and Lane B
       (cross-provider prompt_battery) with explicit guardrails
       against mixing — operator asks for the wrong lane combination,
       runner exits 2 with helpful error.
     - realizer_guard: C1/C2 articulation safety boundary — synthetic
       illegal candidates rejected directly by check_surface AND
       former-bug runtime prompts now produce legal articulations.
     - contemplation (ADR-0080): not under evals/ since it's runtime
       infrastructure that consumes eval reports — contract lives at
       core/contemplation/contract.md.  Documents the read-only +
       SPECULATIVE-only + deterministic-replay invariants and the
       shared DiscoveryCandidateSink plumbing convergence (ADR-0080).

3. evals/CLAIMS.md — Tier 2 rows added

   - frontier_compare Lane A: determinism.primary_score, max_versor_condition
   - frontier_compare Lane B: prompt_battery.primary_score (CORE adapter),
     cross-provider artifact persistence
   - realizer_guard: all_claims_supported
   - contemplation: SPECULATIVE-only invariant, deterministic replay,
     additive sink path, no pack mutation (all CI-pinned by tests)

Verification
------------
$ core test --suite smoke -q
67 passed in 27.22s    (no regression)

$ uv run pytest -q tests/test_contemplation_loop.py \
    tests/test_contemplation_pipeline_convergence.py \
    tests/test_frontier_compare_cross_provider.py
27 passed in 4.87s

$ core bench --suite determinism --json 2>/dev/null | jq .results[0].passed
true        (was: JSONDecodeError on prior [pulse] pollution)

* feat(evals/ui): report viewer renders Lane B cross-provider + pass-rate chart

Stop-hook caught that #62 only covered contracts — the 929-line
report_viewer.html was never audited against the new cross-provider
report shape from #61.  Two real gaps:

1. Lane-aware observation drawer
   The drawer hardcoded Lane A (CORE-native) fields: surface,
   grounding_source, anchor_lens_mode_label, versor_condition.
   Lane B (cross-provider) observations carry different fields:
   provider, model, elapsed_ms, error_type, error_message.

   Loading a cross-provider report rendered only the surface row
   with empty `grounding` — the provider + model + timing data
   was unreachable without expanding "Show raw JSON".

   Fix: detect Lane B (presence of `obs.provider`) and render the
   appropriate field set.  Lane A still renders identically (now
   also surfaces trace_hash + register_id when present, which were
   silently buried in the raw JSON before).

2. Pass-rate chart per suite
   The summary strip showed one aggregate Primary % across all
   suites, with no way to see WHICH suite is dragging the score.
   Multi-suite runs (e.g. --suite all) had to expand each panel
   individually to find the failing one.

   Fix: new .passrate-chart element below the summary strip,
   one horizontal bar per suite showing passed/total.  All-pass =
   solid green, all-fail = solid red, partial = green/red split
   at the pass fraction.  CSS only — no new dependencies.

3. SUITE_PREAMBLES gains the prompt_battery entry so the sidebar
   shows the "side-by-side surface evidence across providers"
   description when loading a Lane B report.

Verified
--------
- Brace/paren/div balance unchanged (308/308 / 380/380 / 54/54)
- One <script> tag pair preserved
- Generated a real Lane B report via
  `python -m evals.frontier_compare --provider core --suite prompt_battery`
  for visual confirmation

Out of scope (noted for future PR)
----------------------------------
Sampled 3 `core demo` targets:
- register-tour: clean schema (all_claims_supported, claims, grid)
- audit-tour: both scene_1_* keys AND an empty scenes:[] array — inconsistent
- anti-regression: no all_claims_supported key, uses all_gates_held instead

Demo schema standardization deserves its own PR — operator tooling
would benefit from a uniform top-level success field across demos.

* docs(evals) + chore(demos): systematic audit + uniform success field

Stop-hook caught two real gaps after the contract+UI PR:
- demos had divergent success-field names (all_gates_held vs
  learning_loop_closed vs claim_supported vs nested claims_supported)
- no systematic look at the 48 eval directories had been done

Both addressed concretely; remaining work captured in audit doc
rather than vaguely deferred.

1. Demo schema standardization — uniform all_claims_supported field
----------------------------------------------------------------------
All 9 ``core demo`` targets now emit a top-level
``all_claims_supported: bool`` field.  Existing per-demo fields
(``all_gates_held``, ``learning_loop_closed``, ``claim_supported``,
nested ``claims_supported``) are preserved for backwards compat —
the new field is an alias derived from the demo's existing success
signal, not a replacement.

Operator tooling and the CI gate can now target
``all_claims_supported`` without knowing each demo's idiomatic
field name.

Files touched:
- evals/anti_regression/run_demo.py — adds AND of all_gates_held +
  active_corpus_byte_identical
- evals/learning_loop/run_demo.py — adds AND of learning_loop_closed +
  active_corpus_byte_identical
- scripts/publish_pack_measurements.py — adds AND of the three
  entries in the nested claims_supported dict
- evals/long_context_cost/comparison_runner.py — adds alias for
  claim_supported (singular)

The 5 demos already using ``all_claims_supported`` (audit-tour,
register-tour, anchor-lens-tour, orthogonality-tour, articulation)
are unchanged.

Verified across all 9 demos:
  audit-tour              : True
  register-tour           : True
  anchor-lens-tour        : True
  orthogonality-tour      : True
  pack-measurements       : True   ← new alias
  anti-regression         : True   ← new alias
  learning-loop           : True   ← new alias
  articulation            : True
  long-context-comparison : True   ← new alias

2. docs/EVAL_AUDIT_2026-05-20.md — systematic 48-lane audit
------------------------------------------------------------
Replaces the "future PR" deferral with a concrete document.

Contains:
- Method (what was inspected for each lane).
- Summary (40/48 have contract.md; 18/48 have saved results;
  empty results/ ≠ broken — most lanes regenerate on demand).
- Cross-provider relevance triage:
    * 9 lanes are cross-provider-relevant and could benefit
      from the prompt_battery-style adapter pattern (cognition,
      english_fluency_ood, hebrew_fluency, koine_greek_fluency,
      grammatical_coverage, inference_closure, multi_step_reasoning,
      discourse_paragraph, foundational_*_ood, etc.).
    * 29 lanes are CORE-only by design (versor closure, anchor
      lens, identity divergence, provenance, etc.) — wiring
      providers would be category-erroneous.
- Demo schema standardization status (this PR closes that).
- UI/UX coverage matrix.
- 5 concrete follow-up items, each focused enough for a single
  PR, none requiring architectural change.

Regenerated reports
-------------------
evals/long_context_cost/results/comparison_v1.json and
evals/results/phase2_pack_measurements.json now contain the new
all_claims_supported field (auto-regenerated when validating the
schema change).

evals/frontier_compare/results/sample_core_promptbattery.json
added as a reference Lane B report so the new viewer always has
something to load on first open.
2026-05-20 13:53:13 -07:00
Shay
db39a5aac7
chore(adr): rename ADR-0081 frontier provider adapters → ADR-0082 (#59)
Resolves a same-day numbering collision: the prior session produced
ADR-0080 + ADR-0081 (geometric stress field, falsified) in
docs/decisions/ while the frontier-provider-adapters work was
authored as ADR-0081 in a newly-created docs/adr/ directory,
unaware of the concurrent track.

This commit takes the minimum-blast-radius fix:
  - docs/adr/ADR-0081-...md → docs/adr/ADR-0082-...md
  - Update title header to ADR-0082, add "Renumbered from" breadcrumb
  - Update the two source-file docstrings that cite the ADR number
    (providers.py, model_registry.py)

The "two ADR directories" question (docs/adr/ vs docs/decisions/)
is NOT resolved here — docs/adr/ now has exactly one entry, while
docs/decisions/ is the canonical location per CLAUDE.md.  A future
PR should either consolidate or document the split; this commit
just unblocks the immediate naming collision.

Out of scope:
  - Consolidating directories
  - Renumbering anything in docs/decisions/
  - Re-numbering on the dev's local main (already pulled into this branch)
2026-05-20 12:46:13 -07:00
Shay
36904369ee feat(evals): ADR-0081 frontier provider adapters — .env.example, providers, model registry 2026-05-20 12:35:34 -07:00
Shay
06bbac86e1
feat(contemplation): ADR-0080 read-only speculative loop (#55)
* docs(adr): ADR-0080 contemplation loop boundary

* feat(contemplation): add read-only contemplation package

* feat(contemplation): add immutable speculative finding schema

* feat(contemplation): add deterministic substrate snapshot

* feat(contemplation): add frontier report miner package

* feat(contemplation): mine frontier compare failures as speculative findings

* feat(contemplation): add read-only contemplation runner

* feat(contemplation): add read-only contemplation CLI

* test(contemplation): prove read-only speculative loop invariants
2026-05-20 11:40:12 -07:00
Shay
9e791045eb docs(adr): ADR-0078 status Proposed → Ratified
Phase 1 telemetry shipped in 8e96728; bump the ADR status header to
match reality.
2026-05-20 06:30:48 -07:00
Shay
e550c4a240
docs(adr): ADR-0078 composer graph atom equivalence (#50) 2026-05-20 06:30:09 -07:00
Shay
8e96728009 feat(telemetry): ADR-0078 Phase 1 — composer/graph atom equivalence (observational)
Wires observational telemetry on the composer-vs-graph atom-set
relationship.  Phase 1 is strictly observational: no enforcement,
no surface mutation, no grounding-source change, no trace-hash impact.

New telemetry fields on TurnEvent + ChatResponse:
  composer_graph_atom_status         ∈ {equivalent, divergent,
                                         graph_unconstrained,
                                         composer_no_atoms,
                                         not_applicable, ""}
  composer_atom_set_hash             SHA-256 over sorted unique atoms
  graph_atom_set_hash                SHA-256 over sorted unique atoms
  composer_graph_atom_overlap_count  int

Composer atoms come from existing pack candidate metadata
(pack_semantic_domains channel through _maybe_pack_grounded_surface).
Graph atoms come from build_graph_from_input + resolve_lemma on
node.subject/predicate/obj — no prose parsing.  When a grounded
composer path lacks explicit atom provenance, status is
'composer_no_atoms'.

New pure helper:
  chat/atom_equivalence.py — normalize_atoms, hash_atoms,
  atoms_for_graph_nodes, compare_atom_sets

Tests (tests/test_composer_graph_atom_equivalence.py):
  - Pack DEFINITION path produces observable equivalence
  - Divergent atom sets produce distinct hashes
  - Register invariance: atom hashes + status identical across
    {neutral, terse, convivial}; trace_hash also constant (R5 axis)
  - Anchor lens engaged case still ASCII-only on surface
  - No prose-parsing helper symbols introduced in runtime.py
    (extract_candidate_surface_lemmas, surface_lemma,
    parse_surface_atoms) — enforces Phase 1 boundary

Performance note: build_graph_from_input now runs on every warm
English turn (previously only when forward_graph_constraint=True).
Phase 1 accepts this cost to make the telemetry universally
available; Phase 2+ can introduce a feature flag if needed.

Validation:
  - Cognition eval byte-identical: 100/100/91.7/100
  - Full lane: 2864 passed, 3 skipped, 0 failed (+5 over baseline)
  - Targeted lane: 72 passed in tests/test_{graph_constraint,
    pack_grounding,register_tour_demo,anchor_lens_tour_demo,
    orthogonality_tour_demo,realizer_guard_holdout,
    composer_graph_atom_equivalence}.py
2026-05-20 06:14:25 -07:00
Shay
5a78b0e37b feat(register): ADR-0077 — substantive register knobs + layering boundary (R6)
R5 (ADR-0072) shipped the register *machinery*; ADR-0074's orthogonality
tour proved the axis was decoratively orthogonal to anchor-lens but
inspection of the cognition-eval surfaces revealed two structural gaps:

* On pack-grounded DEFINITION/RECALL/COMPARISON composers, the only
  realizer override any register consumed was `disclosure_domain_count`
  — which only fires on the no-gloss disclosure path.  Under terse_v1,
  every gloss-DEFINITION cell was byte-identical to default_neutral_v1.
* The register-tour's `surfaces_vary_at_least_once` gate could be
  satisfied by convivial's decorative wrapper alone, masking that
  regression in CI.

R6 closes both:

Layering separation (the load-bearing fix):
* New TurnEvent/ChatResponse field `register_canonical_surface` carries
  the composer output BEFORE any register transformation.  The pipeline
  hashes this field for `trace_hash`, preserving R5's invariant that
  per-prompt trace_hash is CONSTANT across registers even while
  substantive transforms produce visibly different surfaces.

Substantive transforms (`chat/register_substantive.py`):
* terse_v1 gains 3 bool knobs: `drop_provenance_tag`, `compress_gloss`,
  `drop_articles` — all pure regex transforms on the canonical surface.
* convivial_v1 gains `append_semantic_domain_clause` — appends a single
  bounded "Related: <atom>." clause using the lemma's pack atoms.
* default_neutral_v1 leaves overrides empty; substantive transform is
  byte-identical no-op (preserves `byte_identity_null_lift`).
* C1 (ADR-0075) safety preserved: drop_articles refuses to drop
  articles following `not` (avoids R3 violations); no knob combination
  trips R2/R3.

Strengthened tour gate (`evals/register_tour/run_tour.py`):
* Replaces `surfaces_vary_at_least_once` with two falsifiable claims:
  - `terse_substantively_differs_from_neutral_on_pack_grounded_definition`
  - `convivial_substantively_differs_from_neutral_on_pack_grounded_definition`
  Both restrict to DEFINITION+pack-grounded cells and require
  difference beyond whitespace/punctuation.
* New claim `register_canonical_surfaces_identical` directly proves
  the layering separation.
* Preserves R5's `all_grounding_sources_identical` +
  `all_trace_hashes_identical`.

Pack ratification:
* Loader widened to accept `bool` for closed-set R6 keys
  (drop_provenance_tag / compress_gloss / drop_articles /
  append_semantic_domain_clause).
* `_KNOWN_OVERRIDE_KEYS` ratify gate extended with same.
* terse_v1 + convivial_v1 reratified with new knobs; companion
  mastery reports re-sealed.  default_neutral_v1 unchanged.

Invariants pinned:
* `invariant_register_canonical_surface_constant_across_registers` (new)
* `invariant_terse_substantively_distinct_from_neutral` (new)
* `invariant_convivial_substantively_distinct_from_neutral` (new)
* `invariant_realizer_no_illegal_articulation` (C1, preserved)
* `invariant_realizer_guard_byte_identity_on_currently_passing_cases`
  (C1, preserved)

Verification:
* `core eval cognition`: 100.0% / 91.7% / 100.0% / 100.0% — byte-
  identical under default_neutral_v1.
* `core demo register-tour`: all 5 claims green, exit 0.
* `core demo anchor-lens-tour`: green (no anchor-lens code touched).
* `core demo orthogonality-tour`: green (5/5 claims).
* Full lane: 2858 passed, 1 pre-existing failure
  (test_all_preamble_explains_combined_run, carried forward
  unchanged from main).  56 new R6 tests across three files.
2026-05-19 23:39:11 -07:00
Shay
d7499c80b3
feat(intent): normalize confirmation-tag propositions (#45) 2026-05-19 22:55:28 -07:00
Shay
7cc2888ed2 feat(coherence): ADR-0075 — realizer slot-type guard (C1)
C1 coherence floor: a deterministic verifier that runs on every
candidate surface produced by the truth path, before assignment to
ChatResponse.surface.  Rejects illegal articulations and routes them
to a bounded disclosure string — admission control with a
deterministic fallback, not normalization.

Active rules (R1 deferred during ratification — see ADR):
  R2_aux_neg_requires_verb     — "<aux> not <wrong-POS>"  rejected
  R3_be_neg_requires_predicate — "<be>  not <verb>"       rejected

Fail-open on unknown POS, fail-closed on explicit wrong POS.
Cognition eval byte-identical (100/91.7/100/100).

Original bug class — "Light reveals truth, right?" → "Right does not
thought." — now routes to "I do not have a reviewed articulation for
that yet." with grounding_source=none, walk_surface preserving the
rejected candidate, and telemetry carrying R2_aux_neg_requires_verb.

Files:
  generate/realizer_guard.py            NEW — pure verifier
  chat/runtime.py                       hook on stub + main paths
  chat/telemetry.py                     serialize guard fields
  core/physics/identity.py              TurnEvent +2 fields
  evals/realizer_guard/run_holdout.py   NEW — 6-prompt cluster
  tests/test_realizer_guard_*.py        NEW — 46 tests (unit/seam/holdout)
  docs/decisions/ADR-0075-*.md          NEW — ratified

Invariants pinned:
  invariant_realizer_no_illegal_articulation
  invariant_realizer_guard_byte_identity_on_currently_passing_cases

Lanes (excluding 1 pre-existing TestDemoPreambles failure unrelated
to C1, already present at 4426f38):
  smoke 67/67  cognition 120/120(+1s)  teaching 17/17
  packs 6/6   runtime 19/19   algebra 132/132   full 2792/2793
2026-05-19 22:35:09 -07:00
Shay
4426f387d1 feat(demo): ADR-0074 — orthogonality tour (anchor-lens × register)
A single demo that walks the full 3 × 3 × 2 matrix (register × lens
× prompts, 18 cells) and pins five claims simultaneously, packaging
both single-axis invariants into one composition gate.

The single-axis tours assert opposite invariants:

  register-tour    : per (lens, prompt), trace_hash CONSTANT across
                     registers (R5 / ADR-0072).
  anchor-lens-tour : per (register, prompt), engaged lens diverges
                     in trace_hash from the unanchored baseline
                     (L1.4 / ADR-0073d).

Orthogonality-tour packages both claims simultaneously across the
full matrix, plus three surface-level claims that pin the markers
operators actually see.

Composed claims (all five must hold)

  A) inner_register_invariant_within_lens
     For each (lens, prompt) cell, the three register runs share an
     identical trace_hash.  (R5 register-tour, applied 6 times:
     3 lenses × 2 prompts.)

  B) outer_lens_distinctness_within_register
     For each (register, prompt) cell where any non-unanchored lens
     engages, that engaged lens's trace_hash differs from the
     unanchored baseline at the same (register, prompt).
     (L1.4 anchor-lens-tour, applied 6 times: 3 registers × 2 prompts.)

  C) surface_carries_register_marker_under_convivial
     Every convivial cell with a non-empty surface has a non-empty
     register_variant_id.

  D) surface_carries_lens_annotation_when_engaged
     Every engaged cell carries [lens(<id>):<mode>] in surface AND
     a non-empty anchor_lens_mode_label.

  E) no_substrate_glyph_leak_across_grid
     No cell's surface contains Greek/Hebrew/Syriac/Arabic glyphs.
     (ADR-0073c gate re-asserted across the full matrix.)

CLI wiring

  core demo orthogonality-tour            human-readable grid + claims
  core demo orthogonality-tour --json     structured report

Exit code 0 iff all five claims hold.

Files

  evals/orthogonality_tour/__init__.py             NEW
  evals/orthogonality_tour/run_tour.py             NEW
  core/cli.py                                       EDIT
    - cmd_demo handler wires orthogonality-tour
    - demo choices + EPILOG examples updated
  tests/test_orthogonality_tour_demo.py             NEW (9 tests)
  docs/decisions/ADR-0074-orthogonality-tour.md     NEW

Sanity check baked into tests
  test_engaged_cells_appear_for_both_non_trivial_lenses pins that
  grc_logos_v1 engages on knowledge in all 3 registers (3 cells)
  and he_logos_v1 engages on truth in all 3 registers (3 cells).
  Prevents the lift claims being vacuously satisfied by a future
  engagement regression.

Lane evidence

  - 9 new orthogonality-tour tests pass.
  - core demo register-tour      → all_claims_supported: True
  - core demo anchor-lens-tour   → all_claims_supported: True
  - core demo orthogonality-tour → all_claims_supported: True
  - python -m core.cli eval cognition → byte-identical 100/100/91.7/100.
  - Full lane: 2745 passed / 4 skipped / 1 pre-existing failure
    (+9 over L1.4's 2736; the one failure remains
    test_all_preamble_explains_combined_run, unrelated).

No runtime / composer / loader / pack / schema changes.  Pure demo
consumer of existing telemetry contracts.
2026-05-19 20:33:33 -07:00
Shay
1feec74b1c feat(anchor_lens): ADR-0073d — L1.4 telemetry, CLI flag, tour demo
L1.4 closes the anchor-lens inside-out arc (L1.1→L1.4 mirroring
R1→R5).  Substantive axis is now operator-observable,
operator-driven, and demo-falsifiable — exactly what R5 did for
the register subsystem.

Telemetry extension
  - TurnEvent + ChatResponse gain anchor_lens_id +
    anchor_lens_mode_label (both default "" → pre-L1.4
    byte-identical).
  - serialize_turn_event surfaces both fields in every JSONL line.
  - Mode-label extracted via _ANCHOR_LENS_ANNOTATION_RE from the
    PRE-decoration surface (so register decoration cannot interfere
    with anchor-lens telemetry).  Composer remains the sole source
    of truth for engagement; the runtime helper is read-only.

Operator surface
  - core chat --anchor-lens <id> CLI flag threads into
    RuntimeConfig.anchor_lens_id.
  - Invalid id → AnchorLensError caught at cmd_chat and surfaced
    as _die("invalid --anchor-lens pack id: ...", code=2) before
    the REPL launches.
  - Composes with --register (both flags wire through
    _runtime_config_from_args).

Narrative demo
  - evals/anchor_lens_tour/run_tour.py walks 2 prompts × 3
    ratified lenses ({default_unanchored_v1, grc_logos_v1,
    he_logos_v1}).  Asserts four claims:
      * lens_ids_recorded_per_turn
      * trace_hashes_distinct_across_lenses (OPPOSITE of
        register-tour's identical-hash claim)
      * surface_propositions_distinct_across_lenses
      * no_substrate_glyph_leak (block-scoped Greek/Hebrew/
        Syriac/Arabic; stylistic punct allowed)
  - Exit code 0 iff all four hold.
  - Bundled into `core demo` choices + EPILOG.

Tests (30 new)
  - tests/test_anchor_lens_telemetry.py (16) — TurnEvent shape,
    serializer keys, runtime emits per lens / per engagement
    state, ChatResponse mirrors event, mode-label extractor unit.
  - tests/test_anchor_lens_cli.py (9) — _runtime_config_from_args
    threading, invalid id fail-fast, parser flag wiring, parser
    composes with --register.
  - tests/test_anchor_lens_tour_demo.py (9) — four seam claims
    pinned individually + all_claims_supported + per-cell
    anchor_lens_id + unanchored cells empty mode + engaged cells
    carry mode label.

Lane evidence
  - 30 new L1.4 tests pass.
  - core demo anchor-lens-tour --json → all_claims_supported: True.
  - core demo register-tour --json    → all_claims_supported: True.
    Both tours pass simultaneously — orthogonality CI-pinned.
  - python -m core.cli eval cognition → public 100/100/91.7/100
    byte-identical (lens=None / default_unanchored_v1).
  - Full lane: 2736 passed / 4 skipped / 1 pre-existing failure
    (+30 over L1.3's 2706; the one failure remains
    test_all_preamble_explains_combined_run, unrelated).

Live demo (canonical proof)
  P1: 'What is knowledge?'
    default_unanchored_v1  trace=17c9aabe…  mode=(none)
    grc_logos_v1           trace=0198ad4c…  mode=systematic
    he_logos_v1            trace=17c9aabe…  mode=(none)
  P2: 'What is truth?'
    default_unanchored_v1  trace=2557f3e8…  mode=(none)
    grc_logos_v1           trace=2557f3e8…  mode=(none)
    he_logos_v1            trace=ec8d84aa…  mode=covenant-verity

  Engagement is substrate-scoped: grc never touches truth, he
  never touches knowledge.  Trace hashes diverge exactly where the
  lens engages.

Trust boundaries
  - --anchor-lens flag does not bypass ratification; loader still
    enforces companion mastery report self-seal + ratify-time
    substrate-atom existence check (ADR-0073b/c).
  - Mode-label extraction is read-only regex parse; can't forge
    annotations the composer didn't emit.
  - Telemetry stays redact-safe — both fields are identifiers /
    mode labels, not content.  include_content=False emits them
    unconditionally.
  - No new mutation surface; pack files unchanged.

Closes the anchor-lens inside-out arc
  L1.1  content prerequisite                  ✓ (ADR-0073a)
  L1.2  class + loader + unanchored sentinel  ✓ (ADR-0073b)
  L1.3  first lenses + composer wiring        ✓ (ADR-0073c)
  L1.4  telemetry + CLI + tour demo           ✓ (this commit)

  Mirrors the R1→R5 register cadence exactly.  Both axes are now
  operator-observable, CI-falsifiable, audit-traceable, and
  composable via the orthogonality claim pinned in both tours.
2026-05-19 20:21:41 -07:00
Shay
b35bec6465 feat(anchor_lens): ADR-0073c — L1.3 first lenses + composer wiring
L1.3 of the anchor-lens inside-out rollout — first substantive
surface lift on the substantive axis.  Two ratified non-trivial
lenses engage on cognition-pack lemmas via the alignment graph,
appending [lens(<id>):<mode>] annotations to the existing
pack-grounded surface.

Two ratified lenses

  grc_logos_v1 (Greek substrate)
    primary_substrate         : "grc"
    semantic_domain_preferences: ["logos.episteme.systematic_knowledge"]
    cognitive_mode_label       : "systematic"
    Engages on en "knowledge" via grc-core-cog-021 (ἐπιστήμη) →
    en-core-cog-007 alignment edge.

  he_logos_v1 (Hebrew substrate)
    primary_substrate         : "he"
    semantic_domain_preferences: ["logos.aletheia.verity"]
    cognitive_mode_label       : "covenant-verity"
    Engages on en "truth" via he-core-cog-002 (אמת) →
    en-core-cog-002 alignment edge.

  Both ratified under method anchor_lens_lifts_proposition.

Engagement rule (single)

  1. Resolve en_lemma → entry_id (cognition pack).
  2. For each substrate pack matching lens.primary_substrate, load
     alignment.jsonl; find edges where target_id == entry_id.
  3. For each such substrate lemma, if any atom in its
     semantic_domains ∈ lens.semantic_domain_preferences → engage.
  4. No match → None (no annotation; byte-identical surface).

The pivot is shared semantic_domain atoms surfaced via the
alignment graph — exactly the language-neutral commitment from
ADR-0073.  Engagement never touches non-English surface text;
entry_ids and atom strings only.

Surface lift

  no-lens : "Knowledge is X. pack-grounded (en_core_cognition_v1)."
  lens-on : "Knowledge is X. pack-grounded (en_core_cognition_v1) [lens(grc_logos_v1):systematic]."

  Annotation between existing provenance and trailing period.
  Both metadata fields are ASCII-bounded ≤64 chars at the loader
  level, so the annotation can never carry non-ASCII.

Scope deliberately narrow

  L1.3 wiring restricted to pack_grounded_surface /
  build_pack_surface_candidate (DEFINITION/RECALL only).  Other
  composers (COMPARISON / CORRECTION / PROCEDURE / NARRATIVE /
  EXAMPLE / CAUSE / VERIFICATION) accept the anchor_lens kwarg via
  forward-compat default UNANCHORED but do not yet consume it.
  L1.3b or later broadens to those intent shapes.

Ratify gate widening

  Non-null lenses must:
    - have primary_substrate ∈ {grc, he, en}
    - have a non-empty cognitive_mode_label
    - every preferred atom must exist in at least one lemma of the
      named substrate (trust boundary: operators cannot ship a lens
      pointing at atoms not on disk).
  Method: anchor_lens_lifts_proposition.  Null lenses still ratify
  under byte_identity_null_lift (L1.2 method).

Seam allow-list widening

  Truth-path modules (cognition / trace / pipeline / intent /
  propagation / vault / algebra) still refused.  Composer-side
  imports from chat/pack_grounding.py now permitted — the same way
  ADR-0069's R2 widened the register seam.

New invariants pinned (3)

  tests/test_anchor_lens_engagement_unit.py (14 tests) — resolver
  returns mode label only on intended substrate × en lemma pair;
  case-insensitive; engagement None under null lens; synthetic
  lens with unmatched atom returns None; annotation is pure ASCII.

  tests/test_anchor_lens_lifts_proposition.py (17 tests) — grc
  engages on knowledge only, he engages on truth only,
  cross-lens isolation, three-way distinctness, replay determinism
  per (lens × prompt), register-tour seam holds within each lens
  scope (orthogonality CI-pinned, parametrized over 4 lens
  choices).

  tests/test_anchor_lens_no_glyph_leak.py (5 tests) — hard
  block-scoped gate: Greek (U+0370..03FF, U+1F00..1FFF), Hebrew
  (U+0590..05FF), Syriac, Arabic.  Stylistic punctuation
  (em-dash etc.) explicitly allowed; em-dash predates L1.3 by a
  wide margin and is not a substrate-leak risk.  Tested per-lens
  across every cognition case + direct lens-metadata ASCII check.

Lane evidence

  74 anchor-lens tests pass (37 from L1.2 + 37 new).
  python -m core.cli eval cognition → public 100/100/91.7/100
  byte-identical (lens=None / default_unanchored_v1).
  core demo register-tour --json → all_claims_supported: True
  (R5 seam still holds; L1.3 doesn't perturb presentation axis).
  Full lane: 2706 passed / 4 skipped / 1 pre-existing failure
  (+37 over L1.2's 2669; the one failure remains
  test_all_preamble_explains_combined_run, unrelated).

Files

  packs/anchor_lens/grc_logos_v1.json                        NEW
  packs/anchor_lens/grc_logos_v1.mastery_report.json         NEW
  packs/anchor_lens/he_logos_v1.json                         NEW
  packs/anchor_lens/he_logos_v1.mastery_report.json          NEW

  scripts/ratify_anchor_lens_packs.py                        EDIT
    LENS_IDS adds grc_logos_v1 / he_logos_v1; gate widened.

  chat/pack_grounding.py                                     EDIT
    _resolve_anchor_lens_mode, _maybe_append_anchor_lens_annotation,
    _substrate_lexicon_by_entry_id, _en_lemma_to_entry_id.
    build_pack_surface_candidate + pack_grounded_surface gain
    anchor_lens kwarg (default UNANCHORED).

  chat/runtime.py                                            EDIT
    Thread self.anchor_lens into pack_grounded_surface() call.

  tests/test_anchor_lens_pack_seam.py                        EDIT
    Doc-comment updated for L1.3 allow-list.

  tests/test_anchor_lens_*                                   NEW (3 files)

  docs/decisions/ADR-0073c-anchor-lens-composer-wiring.md    NEW
2026-05-19 20:06:02 -07:00
Shay
9b1b63b253 feat(anchor_lens): ADR-0073b — L1.2 class + loader + unanchored sentinel
L1.2 of the anchor-lens inside-out rollout — pack class, loader,
ratified sentinel pack, and runtime threading.  Mirrors the
ADR-0068 register-class pattern exactly.  No composer consumes the
lens yet — that's L1.3.

AnchorLens frozen dataclass (packs/anchor_lens/loader.py)
  - lens_id / version / description / display_name
  - primary_substrate ∈ {grc, he, en, none}
  - semantic_domain_preferences: tuple[str, ...] (ordered, ≤64 atoms
    of ≤64 chars each, no duplicates)
  - cognitive_mode_label: str (≤64 chars)
  - mastery_report_sha256
  - is_unanchored() / is_null_lens() predicates
  - unanchored() classmethod + module-level UNANCHORED singleton

Loader contract (mirror of packs/register/loader.py)
  - safe_pack_id path-traversal rejection
  - Schema validation + envelope bounds checks
  - Companion mastery report self-seal + report_sha256 verification
  - CORE_ALLOW_UNRATIFIED_ANCHOR_LENS=1 dev bypass
  - require_ratified default True
  - No truth-path imports (pinned by seam test)

default_unanchored_v1 ratified pack
  - Null lens: primary_substrate="none", empty preferences,
    empty cognitive_mode_label
  - Self-sealed at b3235072fdbb2219...
  - Ratification method: byte_identity_null_lift
  - scripts/ratify_anchor_lens_packs.py L1.2 gate accepts only
    null lenses; L1.3 will widen.  Idempotent.

RuntimeConfig threading
  - new field: anchor_lens_id: str | None = None
  - new constant: DEFAULT_ANCHOR_LENS = "default_unanchored_v1"
  - ChatRuntime.__init__ loads the lens (None → AnchorLens.
    unanchored(); otherwise load_anchor_lens(id)) and stores as
    self.anchor_lens + self.anchor_lens_id.  Invalid ids fail-fast
    at init via AnchorLensError, not at first turn.
  - No composer reads the attribute yet.

Tests pinned (37 total)
  - tests/test_anchor_lens_pack_loader.py (24) — load happy path,
    sentinel structural identity, invalid id rejection (traversal,
    empty, slashes, missing), ratification bypass paths, companion
    SHA mismatch, bounds (substrate / preferences / atoms / label /
    duplicates / capacity), field-missing, lens_id mismatch with
    filename, unsupported schema_version.
  - tests/test_anchor_lens_null_lift.py (4) — load-bearing L1.2
    invariant `anchor_lens_byte_identity_null_lift`: full public
    cognition lane byte-identical for surface, trace_hash, and
    aggregate metrics between anchor_lens_id=None and
    "default_unanchored_v1".
  - tests/test_anchor_lens_pack_seam.py (9) — AST refuses any
    `packs.anchor_lens` import from truth-path modules (cognition /
    trace / pipeline / intent / propagation / vault / algebra) AND
    refuses any truth-path import from the loader itself.

Lane evidence
  - All 37 anchor-lens tests pass.
  - python -m core.cli eval cognition → public 100/100/91.7/100
    byte-identical (lens loaded but no composer reads it).
  - core demo register-tour --json → all_claims_supported: True
    (R5 seam still holds; L1.2 doesn't perturb register).
  - Full lane: 2669 passed / 4 skipped / 1 pre-existing failure
    (+37 over L1.1's 2632; the one failure remains
    test_all_preamble_explains_combined_run, unrelated).

Trust boundaries (per CLAUDE.md / ADR-0051)
  - safe_pack_id path-traversal rejection at loader entry.
  - No dynamic imports.
  - Loader is read-only; mutation only via ratify script.
  - Seam test refuses any new anchor-lens import upstream of the
    realizer.  L1.3 will widen the allow-list to include composer
    files at the same time it adds composer behaviour — exactly the
    way the register seam was widened at R2.

What L1.2 deliberately does NOT do
  - No composer consumes the lens (that's L1.3).
  - No TurnEvent / ChatResponse telemetry fields (L1.4).
  - No `core chat --anchor-lens` CLI flag (L1.4).
  - No anchor-lens-tour demo (L1.4).
2026-05-19 19:46:34 -07:00
Shay
2dd50b8dc4 feat(packs): ADR-0073a — anchor lens L1.1 content phase
Umbrella ADR-0073 ratified (Accepted); L1.1 content phase
(ADR-0073a) landed.  Pure pack enrichment — no runtime code, no
composer change, no test of behaviour.  Substrate prerequisite for
the L1.2–L1.4 phases.

Greek additions (grc_logos_cognition_v1, 20 → 29 entries)
  Knowledge family (English collapses to `knowledge`):
    - ἐπιστήμη  logos.episteme.systematic_knowledge
    - σύνεσις   logos.synesis.insight
    (γνῶσις at grc-core-cog-007 unchanged — treated as the
     experiential variant by the L1.3 lens config)
  Love family (English collapses to `love`):
    - ἀγάπη   logos.agape.covenant_love
    - φιλία   logos.philia.companion_love
    - ἔρως    logos.eros.passionate_love
    - στοργή  logos.storge.familial_love
  Time family (English collapses to `time`):
    - αἰών    logos.aion.age_era
    - χρόνος  logos.chronos.clock_time
    - καιρός  logos.kairos.opportune_moment

Hebrew additions (he_core_cognition_v1, 20 → 23 entries)
  - חסד    logos.chesed.covenant_loyalty
  - שלום   logos.shalom.wholeness_peace
  - צδק    logos.tzedek.right_order

Alignment.jsonl on both cognition-tier packs (previously only the
micro packs carried alignment)
  - grc_logos_cognition_v1/alignment.jsonl — 20 edges: three-way core
    dyads (word / truth / light / life / beginning / wisdom),
    knowledge-family → en collapse, ἀγάπη↔חסד covenant-love pairing
    (weight 0.86, Septuagintal), `cross_lang.no_english_collapse`
    annotations for love + time families pointing at
    `en-collapse-<family>` sentinel ids (weight 0.0).
  - he_core_cognition_v1/alignment.jsonl — 7 edges: core dyads to en,
    חסד↔ἀγάπη covenant pairing, no-english-collapse annotations for
    חסד / שלום / צδק.

Manifest checksums refreshed per CLAUDE.md doctrine
  - grc_logos_cognition_v1: b45bcf581cee… → 0f9436675707…
  - he_core_cognition_v1:   dee1e8c6ad9a… → 22145d008185…

Design decisions
  - Existing 20 + 20 lemma atoms untouched — downstream tests /
    composers / teaching chains keep referencing the same atoms.
    Only new lemmas carry the distinguishing atoms.
  - `cross_lang.no_english_collapse` edges are metadata not data
    (sentinel target ids, weight 0.0).  Their purpose is letting the
    alignment graph answer "does English split this family?" without
    forcing an artificial English lemma.
  - Every new entry carries `adr-0073a:hand_authored:2026-05-19` in
    its `provenance_ids` so future audits can find the L1.1 cohort
    deterministically.

Verification
  - python -m language_packs verify grc_logos_cognition_v1   → OK
  - python -m language_packs verify he_core_cognition_v1     → OK
  - python -m language_packs compile <both>                  → 29 / 23
    manifold points; spot-check confirms καιρός / צδק resolve.
  - python -m core.cli eval cognition                        → public
    100 / 100 / 91.7 / 100 byte-identical (new lemmas sit on disk but
    no composer references them yet).
  - python -m core.cli test --suite cognition                → 120/1 pass
  - python -m core.cli test --suite smoke                    → 67/0 pass
  - python -m core.cli test --suite full                     → 2632 passed
    / 4 skipped / 1 pre-existing failure (test_all_preamble_explains_
    combined_run rename drift, unrelated).
  - core demo register-tour                                  → exit 0
    (R5 seam still holds; L1.1 doesn't touch register pathway).

What L1.1 deliberately does NOT do
  - No AnchorLens class (that's L1.2 / ADR-0073b).
  - No composer wiring (L1.3 / ADR-0073c).
  - No --anchor-lens CLI flag or demo (L1.4 / ADR-0073d).
  - No teaching corpus in non-English (post-L1).
2026-05-19 19:30:20 -07:00
Shay
f673c0eb06 docs(adr): ADR-0073 — anchor lens substrate (Proposed)
Umbrella ADR for the substantive-variation axis that composes
orthogonally against register (ADR-0068..0072).  Drafted only;
status Proposed.  No code, no pack, no test landed.

Architecture summary
  - Anchor lens is the substantive axis: register varies surface text
    while keeping grounding_source / trace_hash byte-identical;
    anchor lens deliberately moves both because the proposition
    itself changes when the substrate changes.
  - Pivot is shared `semantic_domains` atoms (already on disk across
    grc / he / en cognition packs), not transliteration tables — the
    seam stays language-neutral so future substrates compose without
    touching anchor-lens code.
  - English compound phrasing only at the surface ("knowing-as-
    experience", "knowing-as-system"); Greek / Hebrew glyphs live in
    audit / provenance fields only.  L1.3 invariant
    `anchor_lens_no_glyph_leak` is a hard gate.

Four-phase rollout (mirrors R1–R5 cadence)
  L1.1  content phase — distinction-bearing lemma additions
        (ἐπιστήμη / σύνεσις / ἀγάπη-φιλία-ἔρως-στοργή / αἰών-χρόνος-
        καιρός; חסד / שלום / צδק) + alignment.jsonl on the cognition-
        tier packs.  No code.  Prerequisite for every later phase.
  L1.2  AnchorLens pack class + loader + `default_unanchored_v1`
        sentinel.  Null-lift CI invariant pinned.
  L1.3  First non-trivial lenses (`grc_logos_v1`, `he_logos_v1`)
        wired into chat/pack_grounding.py composers.  Proposition-
        lift invariant + glyph-leak gate pinned.
  L1.4  Telemetry (TurnEvent + ChatResponse gain anchor_lens_id),
        `core chat --anchor-lens` flag, `core demo anchor-lens-tour`
        asserting trace_hashes_distinct_across_lenses (opposite of
        register-tour's claim — both must hold).

Three honest gaps blocking L1.2+
  - Distinction-bearing lemmas absent from cognition packs.
  - No reviewed teaching corpus for non-English (cognition_chains,
    relations_chains, cross_pack_chains all en-only).
  - No realizer infrastructure for cross-lingual surface composition.

L1.1 (pure content) closes all three for the cognition tier.

Orthogonality claim — load-bearing
  register-tour    : per prompt, fix lens, vary register → trace_hash CONSTANT
  anchor-lens-tour : per prompt, fix register, vary lens → trace_hash DISTINCT
  Both must continue to hold; failure of either breaks the seam.
2026-05-19 19:13:01 -07:00