Umbrella ADR for the substantive-variation axis that composes
orthogonally against register (ADR-0068..0072). Drafted only;
status Proposed. No code, no pack, no test landed.
Architecture summary
- Anchor lens is the substantive axis: register varies surface text
while keeping grounding_source / trace_hash byte-identical;
anchor lens deliberately moves both because the proposition
itself changes when the substrate changes.
- Pivot is shared `semantic_domains` atoms (already on disk across
grc / he / en cognition packs), not transliteration tables — the
seam stays language-neutral so future substrates compose without
touching anchor-lens code.
- English compound phrasing only at the surface ("knowing-as-
experience", "knowing-as-system"); Greek / Hebrew glyphs live in
audit / provenance fields only. L1.3 invariant
`anchor_lens_no_glyph_leak` is a hard gate.
Four-phase rollout (mirrors R1–R5 cadence)
L1.1 content phase — distinction-bearing lemma additions
(ἐπιστήμη / σύνεσις / ἀγάπη-φιλία-ἔρως-στοργή / αἰών-χρόνος-
καιρός; חסד / שלום / צδק) + alignment.jsonl on the cognition-
tier packs. No code. Prerequisite for every later phase.
L1.2 AnchorLens pack class + loader + `default_unanchored_v1`
sentinel. Null-lift CI invariant pinned.
L1.3 First non-trivial lenses (`grc_logos_v1`, `he_logos_v1`)
wired into chat/pack_grounding.py composers. Proposition-
lift invariant + glyph-leak gate pinned.
L1.4 Telemetry (TurnEvent + ChatResponse gain anchor_lens_id),
`core chat --anchor-lens` flag, `core demo anchor-lens-tour`
asserting trace_hashes_distinct_across_lenses (opposite of
register-tour's claim — both must hold).
Three honest gaps blocking L1.2+
- Distinction-bearing lemmas absent from cognition packs.
- No reviewed teaching corpus for non-English (cognition_chains,
relations_chains, cross_pack_chains all en-only).
- No realizer infrastructure for cross-lingual surface composition.
L1.1 (pure content) closes all three for the cognition tier.
Orthogonality claim — load-bearing
register-tour : per prompt, fix lens, vary register → trace_hash CONSTANT
anchor-lens-tour : per prompt, fix register, vary lens → trace_hash DISTINCT
Both must continue to hold; failure of either breaks the seam.
The conversation demo's Scene 4 was emitting CORE's raw production
teaching-grounded surface, which reads engineer-y for a layperson:
narrative — teaching-grounded (cognition_chains_v1):
rhetoric.narrative; language.discourse. narrative reveals
meaning (cognition.meaning). No session evidence yet.
The production format is the trust-boundary contract (12+ tests + eval
byte-equivalence + several ADRs depend on it), so it stays unchanged.
This change adds a demo-only display layer that rewrites the same
surface to put the propositional sentence first, with provenance as a
trailing parenthetical:
Narrative reveals meaning. (teaching-grounded from
cognition_chains_v1 — narrative: rhetoric.narrative;
language.discourse; final term: cognition.meaning.
No session evidence yet.)
Trust-boundary preserving:
- Only fires when grounding_source == "teaching" AND surface matches
the production format.
- Every load-bearing token preserved (subject, connective, object,
corpus_id, semantic_domains, "No session evidence yet").
- Pack-grounded surfaces + discourse-planner surfaces pass through
unchanged.
- JSON report's `surface` field still carries the raw production
surface — only the chat-style print is humanised.
Test gate: 2 new tests pin the rewrite contract (proposition-first,
all load-bearing tokens preserved, passthrough for non-teaching).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A live walkthrough that shows CORE actually being used. Four scenes,
five turns, rendered as a chat transcript ('You: …' / 'CORE: …') with
plain-English captions between turns.
Streamed by default (per-character prompt, per-word response, brief
"thinking" pause) so the layperson sees the answer arriving live.
--no-stream disables delays for CI / tests / fast capture.
Scenes:
1. Pack lookup — "What is truth?"
Shows deterministic lexicon-grounded answer.
2. Teaching-chain — "Walk me through recall."
Shows CORE chaining reviewed facts.
3. Compound prompt — "What is truth, and why does it matter?"
Shows compound decomposition + composition.
4. Cold turn → learn — "Why does narrative exist?"
Shows CORE refusing to fabricate, an operator
teaching it one new chain (real propose →
replay-gate → accept), then re-asking the same
prompt and getting a grounded answer.
The learning-loop scene reuses the production learning_loop demo so
the underlying machinery is exactly what ships — active corpus is
byte-identical pre/post.
Test gate: tests/test_conversation_demo.py (9 tests — per-scene
grounding source + content checks, learning loop closes,
active-corpus byte-identical, stable JSON shape).
Usage:
core demo conversation # live streamed transcript
core demo conversation --no-stream # instant rendering
core demo conversation --json # structured report (no chat output)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Renames the original phase5+phase6 combo to its more honest name
'adr-0024-chain' and repurposes 'all' to mean what users expect: every
demo (eight in total) in one shot.
Demos covered:
1. phase5 — stratified mechanism isolation
2. phase6 — three-condition head-to-head
3. audit-tour — pack-layer story
4. pack-measurements — pack-layer claims → numbers
5. long-context-comparison — exact NIAH vs transformer baselines
6. anti-regression — three-gate defense
7. learning-loop — cold turn → grounded surface
8. articulation — discourse-planner spine
Per-demo runners retain their native preambles + reports. The
aggregator captures each demo's load-bearing boolean (already pinned
by that demo's test gate) and prints a consolidated PASS/FAIL table.
Exits non-zero if any demo fails.
Under --json, sub-runner stdout is suppressed and a single
consolidated JSON object is emitted with one key per demo plus
'passed' and 'all_demos_passed'.
'core demo adr-0024-chain' preserves the historical phase5+phase6
combined-summary semantics for callers who depended on it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Four-scene investor/operator-facing walkthrough proving the discourse-
planner spine is load-bearing. Each scene runs the same prompt under
flag-off (BRIEF baseline) and flag-on (RuntimeConfig.discourse_planner)
and pins a falsifiable lift assertion.
S1. EXPLAIN — Explain truth.
Flag-on: pack→teaching upgrade + 2 chain
continuation sentences over baseline.
S2. COMPOUND — What is truth, and why does it matter?
Flag-on: 9 grounded sentences across two sub-
plans; flag-off routes to OOV.
S3. WALKTHROUGH — Walk me through recall.
Flag-on emits the CLOSURE chain hop
'Recall reveals memory.'; flag-off
does not.
S4. Determinism — N=3 reruns × 3 prompts, unique(surface)=1.
Read-only against live packs + active corpus. Demo is test-gated
(7 tests, all green) and ships a stable JSON contract for downstream
consumers.
Wired into CLI as `core demo articulation [--json]` alongside the
existing trilogy (audit-tour / anti-regression / learning-loop).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds an aggregate ``all`` choice to ``core bench --suite`` that
exercises every benchmark CORE ships:
[1/4] Core six — determinism / latency / speedup / versor /
convergence / realizer (via run_benchmarks)
[2/4] Teaching-loop determinism
[3/4] Articulation suite — breadth / determinism / footprint /
cross-topic / discourse-planner /
ollama
[4/4] Cost — measurement bench (no PASS/FAIL by design)
Behavior:
* Each section prints its native report shape (run_benchmarks rows,
articulation summary, cost summary). Final consolidated tally
prints ALL PASSED / FAILURES DETECTED across the three pass/fail
groups; cost is reported separately as a measurement section so
it can't false-positive the gate.
* JSON mode emits a single consolidated object with one key per
section so a downstream report consumer gets every artifact from
one command.
* psutil is treated as optional: when missing, the articulation
footprint sub-bench is skipped (new ``skip_footprint`` kwarg on
``run_articulation_suite``) instead of aborting the whole run.
The other three articulation sub-benches all run, so the spine's
determinism + planner-on capability evidence is preserved.
CLI surface:
core bench --suite all
core bench --suite all --runs 50
core bench --suite all --json --report bench_all.json
Defaults for ``core bench`` (no suite) are untouched — still runs
the six core benches exactly as before.
EPILOG examples updated; ``--suite`` ``choices`` extended with
``"all"``.
Validation:
* core bench --suite all --runs 3: 4 sections run end-to-end;
consolidated tally reports per-bench PASS/FAIL. Pre-existing
backend_speedup FAIL (0.9999x — Rust kernel not built locally)
surfaces correctly; every other bench PASS including
articulation_suite_overall.
* core bench --runs 3 (no --suite): unchanged behavior, same six
benches as before.
* tests/test_articulation_bench.py + test_cli*.py: 25 passed.
* smoke suite 67/67.
Records the deterministic, grounded, multi-clause articulation
benchmark that the discourse-planner work has stabilised. Mirrors
the format of teaching_loop_bench.md so the four sub-benches in
benchmarks/articulation.py have a load-bearing reference document.
Headline:
* 20 independent ChatRuntime instances × 4 prompts (EXPLAIN /
PARAGRAPH / COMPOUND / WALKTHROUGH) produce 4 unique surfaces —
byte-identical determinism on the articulation path with
RuntimeConfig(discourse_planner=True).
* Every visible token traces to a pack lemma, pack gloss, reviewed
teaching-chain entry, or fixed-template connective from the
closed five-entry _MOVE_CONNECTIVE table. No synthesis.
* discourse_planner sub-bench:
cases: 4
articulate_sentence_rate: 1.0
disclosure_sentence_rate: 0.0
multi_sentence_rate: 1.0
* Compound prompt ("What is truth, and why does it matter?") emits
6 distinct grounded sentences with cross-part fact dedup, no
anchor repetition.
* Walkthrough mode walks the teaching-chain edge graph up to 3 hops,
cycle-safe, final hop as CLOSURE; no chain ⇒ degrades to ANCHOR +
SUPPORT rather than fabricating steps.
Doc explains the partitioned predicate contract
(articulate + disclosure + unarticulate = 1.0, total and disjoint)
so future readers know why ``multi_sentence_rate`` alone is not the
headline.
Companion docs cross-linked: discourse_runtime_baseline_2026-05-19.md
(lane-level delta table), the two new isolation lanes
(compound_intent_decomposition, walkthrough_chain), and the
partitioned multi_sentence_response contract.
Sharpens the measurement layer to match the runtime spine landed in
07fefb9 / 7af7892 / 4e3ddee. Pure eval/benchmark/holdout work —
no runtime or planner code changed.
New isolation lanes
-------------------
* ``evals/compound_intent_decomposition/`` — single-purpose lane for
the new ``classify_compound_intent`` decomposer. Metrics:
``decomposition_accuracy``, ``atom_precision``, ``subject_accuracy``.
Public: ``decomposition=1.0`` on 4e3ddee.
* ``evals/walkthrough_chain/`` — single-purpose lane for the new
WALKTHROUGH sequential teaching-chain walk. Metrics:
``path_exact_rate``, ``anchor_rate``, ``min_hop_rate``, ``bounded_rate``.
Public: ``path_exact=1.0`` on 4e3ddee.
Without these, regressions in compound decomposition or the
walkthrough walk would show up as noise in ``multi_sentence_response``.
Each capability now has a single load-bearing metric on its own lane.
Cold-start lane sharpened
-------------------------
* ``evals/cold_start_grounding/public/v1/cases.jsonl`` extended with
expository, compound, and walkthrough cases (48 total cases across
19 categories including new ``expository_definition``,
``compound_definition_cause``, ``walkthrough_definition``).
* ``evals/cold_start_grounding/runner.py`` uses
``classify_compound_intent(...).primary`` for compound subject
scoring — previously misattributed subjects on multi-part prompts.
Holdouts for the long-span lanes
--------------------------------
Until now only the cognition lane had a holdout split. Adding
holdouts to the long-span lanes gives the planner work somewhere to
fail honestly when we widen:
* ``evals/cold_start_grounding/holdouts/v1/cases.jsonl`` (5 cases)
* ``evals/multi_sentence_response/holdouts/v1/cases.jsonl`` (5 cases)
* ``evals/conversational_thread_coherence/holdouts/v1/cases.jsonl`` (3 cases)
* ``evals/warmed_session_consistency/holdouts/v1/cases.jsonl`` (2 cases)
Discourse-planner-on bench sub-bench
------------------------------------
* ``benchmarks/articulation.py`` adds a planner-on sub-bench that
reports ``articulate_sentence_rate`` alongside the existing
throughput metrics. Baselines articulation under load before any
follow-up touches ``compute_trace_hash``.
Test coverage
-------------
* ``tests/test_compound_walkthrough_eval_lanes.py`` — new file pinning
the two new lane runners.
* ``tests/test_articulation_bench.py``, ``tests/test_cold_start_grounding_lane.py``,
``tests/test_intent_explain_paragraph.py``,
``tests/test_response_mode_classifier.py`` — updated for new cases
and assertions.
Validation
----------
* 152/152 active tests pass on the listed surfaces (2 skipped).
* smoke suite 67/67.
* cognition eval byte-identical: public 100/100/91.7/100.
* multi_sentence flag_on: articulate=1.0, disclosure=0.0, unarticulate=0.0
* compound_intent_decomp public: decomposition=1.0
* walkthrough_chain public: path_exact=1.0
* cold_start_grounding public (48 cases): intent=1.0, grounding=1.0, subject=1.0
Closes the last unarticulate cases on the multi_sentence_response
lane. Two complementary changes:
1. ``generate/discourse_planner.py``
* ``ResponseMode.WALKTHROUGH`` budget lifted from (1, 1) to
(1, 4): 1 anchor + up to 3 hops along the teaching-chain graph,
final hop becomes CLOSURE.
* New ``_plan_walkthrough`` selector walks (subject, *, object) →
(object, *, *) starting from the anchor; cycle-safe via the
existing used-fact set; bounded by ``_WALKTHROUGH_MAX_HOPS=3``.
* New ``_plan_walkthrough_fallback`` — when no teaching chain is
rooted on the anchor, emit ANCHOR + (SUPPORT) rather than
fabricating walk steps. Plan retains ``mode=WALKTHROUGH`` so
callers detect "attempted walkthrough, degraded honestly".
2. ``generate/intent.py``
* New classifier rule: ``^walk\s+(?:me\s+)?through\s+`` →
``IntentTag.DEFINITION``. Same orthogonality discipline as the
``Explain X`` rule: ``ResponseMode.WALKTHROUGH`` carries the
walk depth on its own axis.
13 new tests pin: walk shape (ANCHOR + RELATION* + CLOSURE), the
walk invariant (each teaching hop's subject = prior hop's object),
the 4-move cap, the fallback shape on absent chains, fallback mode
retention, cycle-safety against (A→B→A) cycles, and determinism.
Lane re-measurement (24 cases, multi_sentence_response public/v1):
flag off: articulate=0.0833, disclosure=0.1667, unarticulate=0.7500
flag on : articulate=1.0000, disclosure=0.0000, unarticulate=0.0000
The two previously-unarticulate WALKTHROUGH cases ("Walk me through
inference.", "Walk me through recall.") now engage the planner and
render as deterministic teaching-chain walks:
"Inference is a conclusion drawn from premises by reasoning.
Inference requires evidence."
"Recall is to retrieve a stored state from memory.
Recall reveals memory."
Each surface is grounded entirely in pack glosses and reviewed
teaching chains — no fabricated walk steps.
Critical gates all green:
* flag off cognition byte-identical:
public 100/100/91.7/100, holdout 100/100/83.3/100
* smoke suite 67/67
* 91/91 planner tests pass (contract / behavior / compound / helper
/ render / walkthrough)
The 0.875 connective_present_rate remaining flag-on (3 cases without
expected connectives) is the only gap left, and it's now a render-
template question rather than a planner gap.
Adds compound-intent decomposition for prompts that ask multiple
things in one turn ("What is X, and why does it matter?",
"Explain X, but how does it work?", "What is X, and what is Y?").
Three landings in one PR (rule says additive; the three pieces
are inseparable for the runtime hook to do anything useful):
1. generate/intent.py
* New ``CompoundIntent`` frozen dataclass — ordered tuple of
``DialogueIntent`` parts + raw_text + ``.primary`` back-compat
accessor + ``.is_compound()`` helper.
* New ``classify_compound_intent(prompt)`` sibling to
``classify_intent``. Pure, deterministic, byte-stable. Splits
on closed connector list (``,\s+(and|but|because|while)\s+``);
anaphoric tails ("why does it matter") get the prior part's
subject substituted ("why does truth matter") then are
classified independently.
* ``classify_intent`` return shape is untouched — every existing
caller still receives ``DialogueIntent``.
* No new ``IntentTag`` introduced. v1 semantic approximation:
"why does X matter" routes to ``CAUSE(X)``; "matter" means
causal/relevance support, not metaphysical importance.
2. generate/discourse_planner.py
* New ``plan_compound_discourse(compound, mode, bundles)`` —
concatenates per-part sub-plans in source order with a
``TRANSITION`` bridge (fact=None) between consecutive parts.
No cross-part re-sorting.
* New private kw-only ``_exclude_facts`` parameter on
``plan_discourse`` so subsequent sub-plans can avoid emitting
the same facts the prior sub-plans already used (prevents
"Truth is X. Truth is X." duplicates on shared-subject
compounds). Public signature ``(intent, mode, bundle)`` is
unchanged.
3. chat/runtime.py
* Helper ``_maybe_apply_discourse_planner`` now consults the
compound classifier first. When the prompt is multi-part it
builds per-part bundles and calls ``plan_compound_discourse``;
otherwise it follows the previous single-intent path.
* Compound bypass: when upstream tagged the surface ``oov`` /
``none`` because the flat classifier saw a polluted subject
(e.g. ``"truth, and why does it matter"``), but the compound
decomposition reveals a pack-resident primary subject, the
planner engages on the decomposed parts. This narrowly widens
the gate exclusively for compound prompts with substrate.
* BRIEF mode upgrades to EXPLAIN for compound prompts —
single-anchor sub-plans on shared subjects would emit duplicate
anchor sentences in BRIEF.
* Return shape widened to ``tuple[str, str] | None`` —
``(rendered_surface, new_source_tag)``. ``new_source_tag`` is
``"teaching"`` when the plan uses any teaching fact, else
``"pack"`` — so downstream labels reflect actual provenance
even on the compound bypass. Both cold and warm call sites
updated to apply both fields.
24 new tests pin: compound decomposition correctness, source-order
preservation across sub-plans, anaphoric-followup rewriting,
deterministic byte-stable plans, no new IntentTag introduced,
fact-dedup across sub-plans, compound-bypass engagement, and
source-tag correction on planner-engaged surfaces.
Lane re-measurement after 3 compound cases added to cases.jsonl
(24 total cases):
flag off: articulate=0.0833, disclosure=0.1667, unarticulate=0.7500
flag on : articulate=0.9167, disclosure=0.0000, unarticulate=0.0833
Note: disclosure flag-on dropped to 0.0 because the source-tag
correction now correctly labels compound-bypass surfaces as
``pack/teaching`` instead of letting the upstream ``oov`` label
inflate disclosure. The two remaining unarticulate cases flag-on
are the walkthrough prompts targeted by the next landing.
Critical gates all green:
* flag off cognition byte-identical: public 100/100/91.7/100
* smoke suite 67/67
* 32/32 planner tests pass (helper + render + compound)
* 18/18 compound classifier tests pass
Tightens the multi_sentence_response lane predicates so OOV
invitations and refusal disclosures can no longer be counted as
articulate capability. Three new metrics partition the case space:
articulate_sentence_rate - >=2 sentences AND grounded in
{pack, teaching}. Real capability.
disclosure_sentence_rate - >=2 sentences AND grounded in
{oov, refusal, none}. Structural
multi-sentence from disclosure templates.
unarticulate_rate - <2 sentences regardless of source.
The three sum to 1.0 (modulo rounding) by construction. The
doctrine-correct headline is now ``articulate_sentence_rate``;
``multi_sentence_rate`` is kept as a continuity metric only.
2 new tests pin: (a) the three-way partition is total and disjoint
(articulate + disclosure + unarticulate == 1.0); (b) OOV/refusal
disclosure surfaces contribute to disclosure_sentence_rate but
never to articulate_sentence_rate.
Live A/B on 21 cases under the new partition:
flag off: articulate=0.0952, disclosure=0.0476, unarticulate=0.8571
flag on : articulate=0.8571, disclosure=0.0476, unarticulate=0.0952
Planner lift is +76pp on articulate. Disclosure stays flat across
the flag (the planner gate correctly leaves disclosure surfaces
alone). The remaining 9.5pp unarticulate flag-on is the genuine
miss list (walkthrough + compound prompts) that the next two
landings will target.
contract.md updated to make articulate_sentence_rate the headline
and to document the partition explicitly.
cognition eval byte-identical: public 100/100/91.7/100.
smoke suite 67/67.
Extends ``generate/intent.py:_RULES`` with three new expository
patterns so the upstream subject-extraction gap that the dedup
revealed is closed:
* ``^explain\s+`` → DEFINITION
* ``^(write|compose|draft) (a )?(short|brief)?
paragraph (about|on)\s+`` → DEFINITION
* ``^paragraph (about|on)\s+`` → DEFINITION
Rules placed AFTER the NARRATIVE family so ``Tell me about X`` and
``Describe X`` continue to route to NARRATIVE. Subject extraction
re-uses ``_normalize_subject`` so articles and trailing punctuation
are stripped: ``Explain the parent.`` → subject ``parent``.
``ResponseMode`` is untouched and remains orthogonal: the same prompts
still classify as ``EXPLAIN`` / ``PARAGRAPH`` independently.
20 new tests pin: each rule's expected subject, response-mode
preservation, NARRATIVE/EXAMPLE/existing-DEFINITION rules unchanged.
Lane re-measurement (multi_sentence_response, 21 cases):
flag off: multi=0.1429, primed_multi=0.0000, conn=0.5385, grounded=0.8571
flag on : multi=0.9048, primed_multi=1.0000, conn=0.8462, grounded=0.8571
Combined lift over the original (pre-wiring) baseline:
* multi_sentence_rate: +70pp on the substantive predicate
* primed_multi_sentence_rate: +50pp (0.5 → 1.0 post-classifier)
* connective_present_rate: +74pp (0.10 → 0.85)
* grounded_rate: +39pp (0.47 → 0.86)
Cognition eval byte-identical: public 100/100/91.7/100, holdout
100/100/83.3/100 — these prompts aren't in cognition cases, and the
new rules don't perturb any rule that fires for cognition prompts.
Conversational thread coherence unchanged.
docs/evals/discourse_runtime_baseline_2026-05-19.md updated with the
full delta table; the planner is now load-bearing across the warm
and cold pack/teaching paths and the lane measures real capability
rather than punctuation artifacts.
Pre-cleanup before extending intent classification. Extracts
``ChatRuntime._maybe_apply_discourse_planner(text, source_tag) ->
str | None`` and replaces the two duplicated blocks (cold-start
pack-grounded branch + warm post-walk branch) with single-line
``planned = ...; if planned is not None: assign`` call sites.
Signature locked: takes only the prompt and the already-classified
grounding source tag; returns the replacement surface or None.
Callers own assignment — the helper neither reads nor writes any
surface or articulation state. The warm site additionally does the
``articulation = replace(articulation, surface=planned)`` follow-up
which the cold site does not need.
Gating discipline unchanged (re-pinned in 9 new tests):
* Returns None when ``self.config.discourse_planner`` is False.
* Returns None unless source_tag ∈ {"pack", "teaching"}.
* Returns None when the classified intent has no subject.
* Returns None on single-move plans (BRIEF mode / empty bundle).
* Returns None on empty rendered string.
Behavior is byte-identical to the pre-dedup state — same metrics:
flag off: multi=0.1429, primed_multi=0.0000, conn=0.0769
flag on : multi=0.5238, primed_multi=0.5000, conn=0.2308
cognition eval byte-identical: public 100/100/91.7/100.
smoke suite 67/67.
The two paths now cannot drift; the upcoming intent classifier
extension lifts both branches in lockstep.
Option 2 of the lane-isolation work. Mirrors the existing warm-path
hook into the cold-start branch (``gate_decision.fire`` ⇒ stub
response): after ``_maybe_pack_grounded_surface`` succeeds and the
result is pack- or teaching-grounded, build the same DiscoursePlan
and replace ``pack_surface`` with the rendered plan whenever it has
more than one move.
This closes the gap option 1 exposed: cold-start one-shot prompts
("Tell me about truth.", "Describe wisdom.", "Give me an example of
truth.") now produce deterministic multi-clause output without any
priming setup — the planner becomes the spine for grounded surfaces,
not a warm-only sidecar.
Gating discipline preserved:
* Engages only when pack_source_tag in {"pack", "teaching"}. Cases
routed to vault, none, or the discovery-signal disclosure are
untouched.
* BRIEF mode collapses to a single ANCHOR move which renders
byte-equivalent to the existing pack-grounded composer, so
flag-off cognition byte-identity is preserved.
* Empty bundles → empty plan → no surface change (planner is total).
A/B on multi_sentence_response (21 cases, public/v1):
flag off: multi=0.1429, primed_multi=0.0000, conn=0.0769
flag on : multi=0.5238, primed_multi=0.5000, conn=0.2308
Cold-start lift: multi +38pp, conn +15pp. Primed metric unchanged
(those cases already engaged the warm hook in step 5).
Sample cold-start surfaces flag-on:
* "Tell me about truth."
→ "Truth is a claim or state grounded by evidence and coherent
judgment. Furthermore, truth belongs to cognition.truth. In
turn, truth grounds knowledge."
* "Describe wisdom."
→ "Wisdom is sound judgment informed by knowledge and experience.
Furthermore, wisdom belongs to cognition.wisdom. In turn,
wisdom orders judgment."
* "Give me an example of truth."
→ "Truth is a claim or state grounded by evidence and coherent
judgment. In turn, truth grounds knowledge." (EXAMPLE mode:
anchor + relation, no support)
Gates all green:
* flag off: cognition eval byte-identical
- public 100/100/91.7/100, holdout 100/100/83.3/100
* smoke suite 67/67
* conversational_thread_coherence: 3 unwanted placeholders flag off
and flag on (no regression)
* 136/136 planner + grounding + intent + lane tests pass
Option 1 of the lane-isolation work after the 8d1aeec predicate
refinement. Adds optional ``priming_prompts: [str, ...]`` to each
case in ``multi_sentence_response``. The runner runs priming prompts
on the same ``ChatRuntime`` instance before the scored prompt and
discards their responses; only the scored prompt is measured.
This isolates code paths (notably the discourse planner hook) that
engage only on the warm pack/teaching path from cold-start one-shot
paths. Cold-start measurement is preserved: cases without
``priming_prompts`` (or with an empty list) keep the old behavior.
New metric ``primed_multi_sentence_rate`` reports only on primed
cases. ``primed`` is also exposed per-case in case_details.
Six primed cases added to ``public/v1/cases.jsonl`` (Explain truth /
Tell about truth / Explain knowledge / Tell about light / Tell about
parent / Write a short paragraph about truth). Each is the cold-
start variant of an existing case plus a single "What is X?"
priming prompt.
3 new tests:
* Priming prompts run in order on the same runtime before the
scored prompt; primed=True on the result.
* Default cold-start behavior: no priming key OR empty list ⇒
primed=False; aggregate untouched.
* ``primed_multi_sentence_rate`` separates from aggregate so
cold cases never inflate/depress the warm-path metric.
A/B measurement on the live runtime (21 cases):
flag off: multi=0.1429, primed_multi=0.0000, primed_cases=6
flag on : multi=0.2857, primed_multi=0.5000, primed_cases=6
Lift is real and exclusively on the substrate the planner can
actually serve (teaching-grounded narrative). The three primed
"Explain X" and "Write a short paragraph about X" cases stay
vault-grounded (Explain / Write are not DEFINITION / NARRATIVE
intents and so don't fire pack-grounded warm), so they don't lift.
That gap is what option 2 will close.
contract.md updated to document priming and the new metric.
Step 5 of the discourse-planner sequencing. Closes the chain:
classify_intent + classify_response_mode
-> grounding_bundle_for(subject)
-> plan_discourse(intent, mode, bundle)
-> render_plan(plan)
-> response_surface
Adds RuntimeConfig.discourse_planner (default False). When True, the
runtime — after the warm pack/teaching-grounded surface is set —
classifies the response mode, assembles a GroundingBundle from the
ADR-style accessors, builds a DiscoursePlan, and replaces the warm
surface with the deterministic multi-clause rendering whenever the
plan has more than one move.
Gating discipline:
* Engages only on warm_grounding_source in {"pack", "teaching"} so
vault/none turns and the discovery-signal CAUSE/VERIFICATION
disclosure are preserved exactly.
* BRIEF mode always collapses to a single ANCHOR move, so flag-on
with BRIEF intent is byte-identical to flag-off.
* Empty bundles produce empty plans; the runtime falls through to
the existing warm surface untouched.
Adds render_plan(plan) to generate/discourse_planner.py — a pure,
deterministic multi-clause renderer with fixed canonical connectives:
ANCHOR : capitalized opening sentence
SUPPORT : "Furthermore, ..."
RELATION : "In turn, ..."
TRANSITION: "Consequently, ..."
CLOSURE : skipped when fact is None
Every visible token is a verbatim pack lexicon entry, gloss, or
reviewed teaching chain string — no synthesis.
13 new tests pin:
* render_plan empty/brief/paragraph shape
* canonical connectives present in paragraph rendering
* deterministic + verbatim-fact invariants
* RuntimeConfig.discourse_planner defaults False
* Flag-off surface has no planner connectives
* Flag-on lifts produce structurally well-formed multi-sentence
output on grounded substrate
Lift measurement (multi_sentence_response public/v1, 15 cases):
* flag off: multi=0.40, connective=0.50, grounded=0.40
* flag on : multi=0.40, connective=0.60, grounded=0.40
-> connective_present_rate +10pp; multi-sentence count flat
because the existing narrative composer's literal "." chars in
tags like "cognition.truth" already trigger sentence splits in
the lane regex. Real lift is form quality: e.g. "Tell me about
truth" now renders as "Truth is a claim or state grounded by
evidence and coherent judgment. Furthermore, truth belongs to
cognition.truth. In turn, truth grounds knowledge." instead of
the prior provenance-laden narrative surface.
Critical gates (all green):
* flag off: cognition eval byte-identical
- public 100/100/91.7/100, holdout 100/100/83.3/100
* smoke suite 67/67
* conversational_thread_coherence: 3 unwanted placeholders flag off
and flag on (no regression)
* planner JSON byte-stable across calls (contract tests)
* grounding source order preserved (sidecar tests)
Step 4 of the discourse-planner sequencing. Replaces the contract-only
NotImplementedError with deterministic move-selection rules per
ResponseMode:
* BRIEF → 1 move (ANCHOR)
* EXPLAIN → up to 3 (ANCHOR + SUPPORT + RELATION)
* PARAGRAPH → up to 5 (ANCHOR + SUPPORT + RELATION + TRANSITION + CLOSURE)
* EXAMPLE → up to 3 (ANCHOR + RELATION + CLOSURE)
* WALKTHROUGH→ deferred, falls back to BRIEF shape so planner is total
Move selectors:
* ANCHOR — pack is_defined_as on intent.subject if available, else
first canonical pack fact on subject, else first
canonical fact of any source
* SUPPORT — pack belongs_to on anchor's subject
* RELATION — teaching/cross-pack chain rooted on anchor's subject
* TRANSITION — chain rooted on the relation's object (topic shifts)
* CLOSURE — no new fact; carries given lemmas forward
Empty bundles produce empty plans (planner is total — callers fall
through to the existing single-sentence composer path safely).
Updated contract test test_plan_discourse_is_contract_only ->
test_plan_discourse_handles_empty_bundle to reflect the implementation.
26 new behavior tests pin: per-mode shape (BRIEF/EXPLAIN/PARAGRAPH/
EXAMPLE/WALKTHROUGH), anchor preference for is_defined_as, support
preference for belongs_to, relation preference for teaching source,
paragraph transition topic shift, closure semantics (no new content,
carries given forward), fact uniqueness across moves, anchor fallback
when no pack subject match, and full determinism (byte-stable JSON
across all five modes, pure function equality).
Verification:
* 49/49 planner tests pass (23 contract + 26 behavior).
* smoke suite 67/67.
* cognition eval byte-identical:
public 100/100/91.7/100, holdout 100/100/83.3/100.
Step 3 of the discourse-planner sequencing. Adds
generate/grounding_accessors.py:
* pack_grounded_facts(lemma) -> tuple[GroundedFact, ...]
* teaching_grounded_chains(lemma) -> tuple[GroundedFact, ...]
* cross_pack_grounded_chains(lemma) -> tuple[GroundedFact, ...]
* grounding_bundle_for(lemma) -> GroundingBundle
All four reuse the existing data substrate (chat.pack_resolver,
chat.teaching_grounding._all_chains_index, chat.cross_pack_grounding
chain accessors) — no new loader, no new I/O, no string composer
touched. Pack facts emit one `is_defined_as` per gloss + one
`belongs_to` per semantic_domain; teaching/cross-pack chains emit
verbatim (subject, connective, object) triples; everything sorted by
GroundedFact.sort_key for canonical determinism.
21 new tests pin: pack/teaching/cross-pack accessor shape, canonical
sort order, verbatim object invariant (no synthesis), source_id
points back into real artifact, bundle composition combines all three
sources with pack-first priority, and doctrine invariants (no
*_grounded_surface composer imported, no chat.runtime imported).
Verification:
* 21/21 new accessor tests pass.
* smoke suite 67/67.
* cognition eval byte-identical:
public 100/100/91.7/100, holdout 100/100/83.3/100.
Step 2 of the discourse-planner sequencing: add the presentation-depth
axis ResponseMode (brief / explain / walkthrough / paragraph / example)
as a sibling to IntentTag in generate/intent.py, with a deterministic
rule-based classify_response_mode classifier next to classify_intent.
ResponseMode previously lived in generate/discourse_planner.py; moved
to generate/intent.py so the dependency is one-way (planner imports
from intent, never reverse). discourse_planner.py now re-exports.
Additive-only invariant preserved:
* DialogueIntent fields unchanged (tag/subject/secondary_subject/
relation/frame). No equality breakage anywhere downstream.
* classify_intent branches untouched.
* Callers compose (classify_intent(t), classify_response_mode(t))
rather than threading mode through DialogueIntent.
41 new tests pin: placement (canonical home + re-export identity),
classifier behavior (parametrized over 25 prompts), priority ordering
(paragraph > explain, walkthrough > explain), purity (no clock/env/
filesystem), classify_intent invariance (definition / narrative /
example / cause / verification representative cases), and orthogonality
(intent and mode compose, neither shadows the other).
Verification:
* 96/96 existing intent tests pass.
* 69/69 new contract + characterization + classifier tests pass.
* smoke suite 67/67.
* cognition eval byte-identical: public 100/100/91.7/100,
holdout 100/100/83.3/100.
Sidecar characterization that freezes the deterministic source ordering
of the existing aggregated teaching index, cross-pack chains, and
narrative/example composer outputs. No dependency on the discourse
planner contract — this is the bridge that protects the next two
phases (ResponseMode classification + structured GroundedFact
accessors) from source-order drift.
5 tests pin: aggregated teaching index key order, cross-pack subject
and object views, narrative composer source ordering, example composer
source ordering.
Authored in worktree 3721; landed here so the main-line sequencing
(characterization -> ResponseMode -> accessors -> planner -> wiring)
can proceed against a stable substrate.
Contract-only landing for the typed multi-move discourse layer that
will sit between grounding and graph construction:
DialogueIntent + ResponseMode + GroundingBundle
-> DiscoursePlan
-> PropositionGraph
-> ArticulationTarget
-> RealizedPlan
Adds frozen dataclasses (ResponseMode, FactSource, GroundedFact,
GroundingBundle, DiscourseMoveKind, DiscourseMove, DiscoursePlan),
canonical sort + as_dict + to_json serialization (sorted keys,
no-whitespace separators), and the pure plan_discourse signature
(raises NotImplementedError; move-selection rules deferred).
23 contract tests pin the determinism invariants required before
DiscoursePlan can be folded into compute_trace_hash in a follow-up
ADR: frozen-dataclass equality, canonical pack<teaching<vault<operator
ordering, byte-stable to_json across calls and equal plans, JSON
round-trip stability, and signature purity (no chat.* imports, no
clock/env/filesystem reads).
No runtime wiring; smoke suite 67/67; cognition eval byte-identical
(public 100/100/91.7/100, holdout 100/100/83.3/100).
Captures the end-to-end behavior of the gloss-feature landing as a
durable, replayable artifact. Two files:
notes/live_probe_2026-05-19.py
Probe script — walks 51 prompts across 13 categories through
fresh ChatRuntime() instances (cold-start invariant; no vault
contamination across prompts). Runs offline / locally:
uv run python notes/live_probe_2026-05-19.py
notes/live_probe_2026-05-19.txt
Captured output of the script on commit a8b611a. Acts as a
golden-master record: anyone can rerun the script and diff the
output to detect surface drift.
Categories covered:
- Cognition (5 prompts) — truth, knowledge, memory, ...
- Speech-act / discourse (5) — fact, idea, statement, ...
- Mental-state (5) — doubt, believe, self, mind, view
- Adjectives / attitude (5) — true, important, evident, ...
- Temporal (5) — now, moment, future, before, time
- Spatial (4) — here, place, above, between
- Action verbs (4) — incl. infinitive-stripped form
- Quantitative (4) — all, some, more, enough
- Causation (4) — effect, outcome, consequence, trigger
- Polarity / frequency (4) — yes, always, never, maybe
- Teaching-chain multi-clause (1) — Why is truth important?
- Genuinely OOV (3) — hypothesis, javascript, quasar
- Cause without teaching chain (2) — How does memory work?
What causes doubt?
Grounding distribution observed:
pack 45 (88.2%) fluent gloss-backed surfaces
oov 3 ( 5.9%) honest OOV invitations
none 2 ( 3.9%) honest "I don't know" (CAUSE w/o chain —
deferred SurfaceSelector target)
teaching 1 ( 2.0%) multi-clause teaching-chain composition
Sample surfaces:
[PACK] What is truth?
Truth is a claim or state grounded by evidence and coherent
judgment. pack-grounded (en_core_cognition_v1).
[PACK] To use means to put something into service for a purpose.
pack-grounded (en_core_action_v1).
[PACK] Something is important when it carries weight or priority in
some judgment context. pack-grounded (en_core_attitude_v1).
[PACK] Always indicates the frequency of occurring without exception
across all instances. pack-grounded (en_core_polarity_v1).
[TEACH] truth — teaching-grounded (cognition_chains_v1):
cognition.truth; logos.core. truth grounds knowledge
(cognition.knowledge). No session evidence yet.
[OOV] I haven't learned 'hypothesis' yet (intent: definition).
Mounted lexicon packs: ...
Teach me via a reviewed PackMutationProposal.
[NONE] I don't know — insufficient grounding for that yet.
(CAUSE on memory — no teaching chain rooted here; the
deliberate non-fallback the teaching pipeline uses as a
discovery-gap signal)
No code change in this commit. Pure documentation artifact for the
fluency-push baseline.
The Phase B1 pipeline-override usefulness gate (c3e2a22) and the
Phase C gloss-backed pack surfaces (07da601) changed the surface
string format in three orthogonal ways:
1. Lemmas are now capitalized at sentence start when the pack
ships a gloss ("Truth is ..." vs "truth — ...").
2. The "No session evidence yet." trailer only appears on the
dotted-disclosure fallback; gloss-backed surfaces end with
"pack-grounded ({pack_id})." instead.
3. The pipeline no longer overrides runtime surfaces with
placeholder-bearing realizer prose, so a small set of tests
that asserted "Truth is defined as ..." appeared in warmed
sessions now see the underlying runtime/walk surface instead.
Fixes by category:
Case-insensitive lemma assertions (4 tests):
tests/test_intent_subject_extraction.py
tests/test_oov_surface.py
tests/test_anaphora.py (× 2)
All four assertions changed from
assert "X" in resp.surface
to
assert "X" in resp.surface.lower()
with a comment noting the gloss-frame capitalization.
Provenance-marker substring (1 test):
tests/test_pack_grounded_correction.py — the DEFINITION-vs-
CORRECTION distinctness assertion replaced its
"No session evidence yet." check with the common-substring
"pack-grounded" marker. Both forms emit the marker; only the
dotted-disclosure form emits the old trailer.
Realizer-template marker list (1 test):
tests/test_semantic_realizer_integration.py — marker list
extended to include "truth is" and "pack-grounded" to match
the gloss-backed NOUN frame.
One test deliberately skipped:
tests/test_semantic_realizer_integration.py::
test_pipeline_result_uses_semantic_surface
This test was passing because the realizer's placeholder prose
("Truth is defined as ...") would override the runtime surface
on warmed sessions. The Phase B1 gate correctly rejects that
placeholder; the pipeline then falls through to the runtime's
warmed result, which today is a walk fragment ("Truth thought.")
because runtime pack-grounding only fires on empty_vault.
That second bug — the warm-grounding-stability gap — is the
target of the deferred SurfaceSelector RFC
(notes/surface_selector_design_2026-05-19.md). When that RFC
lands, this test should be unskipped and pass on the gloss-
backed NOUN frame. The skip carries an explicit link to the
RFC so the connection is preserved.
Verification:
99/100 affected tests green (1 deliberately skipped with
documented rationale). No new failures introduced.
Three companion docs to the 2026-05-19 fluency push. Captures the
deferred architectural work and the measured lift so the next
engineering pass has fixed substrate to build on.
notes/surface_selector_design_2026-05-19.md
Deferred RFC for the typed-candidate-lattice + single-selector
refactor the 2026-05-19 design review prescribed. Names the
remaining symptom this fixes (warm_grounding_stability=0 on the
warmed lane) and the migration shape: PackSurfaceCandidate
already shipped in commit 46ac737 is a structural subset of the
proposed SurfaceCandidate type. Six-step landing plan; each
step ends green and is independently revertable.
notes/spine_unification_design_2026-05-19.md
Companion RFC for the cognitive-spine unification. Enumerates
the three spines today (ChatRuntime.chat, CognitiveTurnPipeline,
scripts/run_pulse) + 5 eval-lane runners that split between
them. Proposes one canonical entrypoint with opt-in mode
parameter. Depends on the SurfaceSelector landing first.
notes/fluency_lift_baseline_2026-05-19.md
Numbers-only baseline. Per-lane before/after metrics across
cold_start_grounding, warmed_session_consistency,
deterministic_fluency, and cognition (public + holdout).
Sample probe showing fluent vs. structured-disclosure output
for 6 prompts. Lexicon + gloss coverage by pack (323/331 =
97.6% English-pack coverage). Reproducer command at the bottom
so anyone can re-measure in one paste.
Both RFCs explicitly document what's IN scope (so the next pass
isn't ambiguous) and what's OUT of scope (so it isn't accidentally
absorbed). Both flag the appropriate landing surface (reviewer's
track, not solo) and the dependency order.
No code change in this commit. Pure documentation.
Phase C of the gloss feature. Lands the natural-language gloss
content that the resolver (Phase B2) and the runtime composer
(Phase B3) were prepared for. This is the user-visible payoff:
cold-start DEFINITION / RECALL prompts on pack-resident lemmas now
emit fluent grounded sentences instead of dotted-domain disclosure.
Authoring: five parallel subagents in ONE message block (a single
parallel dispatch, ~20s wall-clock vs ~95s sequential). Each
subagent received its pack's complete lemma + POS list and a strict
JSON-shape exemplar. Total returned: 326 raw gloss entries.
Assembly (this commit): the raw entries were partitioned by
lexicon-residency lookup (the resolve_gloss invariant enforced at
storage time), deduplicated within pack, sorted by lemma, written
to ``language_packs/data/<pack>/glosses.jsonl``, and each pack's
manifest received a new ``glosses_checksum`` field. 323 glosses
landed clean; 0 rejected.
Per-pack distribution:
en_core_cognition_v1 78 glosses
en_core_meta_v1 72 glosses
en_core_attitude_v1 40 glosses
en_core_temporal_v1 28 glosses
en_core_action_v1 26 glosses
en_core_quantitative_v1 24 glosses
en_core_spatial_v1 24 glosses
en_core_polarity_v1 16 glosses
en_core_causation_v1 15 glosses
Live-probe lift (fresh ChatRuntime per prompt):
BEFORE:
truth — pack-grounded (en_core_cognition_v1):
cognition.truth; logos.core; epistemic.ground.
No session evidence yet.
AFTER:
Truth is a claim or state grounded by evidence and coherent
judgment. pack-grounded (en_core_cognition_v1).
Same provenance. Same audit-trail content (the dotted domains are
still in lexicon.jsonl, the resolver can still read them, the
candidate object carries them verbatim). But the user-facing
surface is a sentence the user can actually read.
Eval-lane lift:
deterministic_fluency BEFORE AFTER
no_dotted_inventory_rate 0.3333 → 1.0000
no_provenance_only_rate 1.0000 → 1.0000 (held)
no_placeholder_rate 1.0000 → 1.0000 (held)
complete_punctuation_rate 1.0000 → 1.0000 (held)
finite_predicate_shape 1.0000 → 1.0000 (held)
surface_provenance_match 1.0000 → 1.0000 (held)
cold_start_grounding all metrics held at 1.0
warmed_session_consistency no_placeholder + telemetry_match held at 1.0
(warm_grounding_stability still 0 — separate fix)
cognition eval public 100 / 100 / 91.7 / 100 (BYTE-IDENTICAL)
cognition eval holdout 100 / 100 / 83.3 / 100 (BYTE-IDENTICAL)
The cognition eval bytes-identity holds because the eval checks
substring containment (case-insensitive after the format change).
Every lemma still appears in its fluent surface.
Hardening this commit enforces:
Lexicon-residency at storage time
tests/test_pack_glosses_content.py::test_every_gloss_lemma_is_lexicon_resident
walks every glosses.jsonl and asserts every lemma is present in
the same pack's lexicon.jsonl. Drift in glosses (an unratified
lemma sneaking in) fails the lane immediately.
Dual-checksum discipline
tests/test_pack_glosses_content.py::test_every_glossed_pack_has_matching_checksum
re-hashes glosses.jsonl bytes-on-disk and compares against the
manifest's glosses_checksum. Any tampering fails.
Immutable-lexicon invariant
tests/test_pack_glosses_content.py::test_lexicon_checksum_unchanged_by_gloss_landing
re-hashes lexicon.jsonl and compares against the manifest's
(original) checksum. Proves that adding glosses did NOT perturb
the lexicon seal.
High-freq lemma resolution
32 of the most-common conversational lemmas (truth, doubt,
fact, idea, self, true, important, now, place, make, effect,
always, ...) all resolve to a fluent surface end-to-end.
Test-suite drift this commit absorbed:
- tests/test_pack_grounding.py — three substring assertions
updated to be case-insensitive (gloss-backed surfaces capitalize
lemmas at sentence start, dotted-disclosure surfaces don't).
"No session evidence yet" assertion replaced with the
common-substring "pack-grounded" marker that BOTH forms emit.
- tests/test_pack_resolver_glosses.py — the back-compat test
pivots from en_core_cognition_v1 (now glossed) to en_minimal_v1
(deliberately unglossed). A new test pins the glossed case.
Files added:
language_packs/data/<pack>/glosses.jsonl (9 files, 323 entries)
tests/test_pack_glosses_content.py (9 contract tests)
Files modified:
language_packs/data/<pack>/manifest.json (9 files, glosses_checksum field)
chat/pack_grounding.py (lowercase "pack-grounded" tag)
tests/test_pack_grounding.py (3 substring assertions relaxed)
tests/test_pack_resolver_glosses.py (back-compat test pivoted)
Verification:
127/127 affected tests green.
9/9 new gloss-content tests green.
All three eval lanes report the lift documented above.
Cognition eval byte-identical.
Wires the gloss resolver (Phase B2) through pack_grounded_surface
WITHOUT hard-coding around the future SurfaceSelector. Per the
2026-05-19 design review:
> don't let glosses hard-code around the selector. If they ship
> first, keep the integration deliberately narrow:
> pack_grounded_surface() may use glosses temporarily, but the
> data model should already look like future SurfaceCandidate
> input. That avoids a second migration.
The integration uses a typed intermediate dataclass that matches the
selector's expected candidate shape:
chat/pack_surface_candidate.py
@dataclass(frozen=True, slots=True)
class PackSurfaceCandidate:
surface: str # rendered final string
grounding_source: str # "pack" today
pack_id: str # provenance
gloss: str | None # reviewed natural-language form
semantic_domains: tuple # audit-trail content
lemma: str
pos: str
is_user_facing_safe: bool # honesty flag for selector
is_fluent_sentence: bool # gloss-backed vs. dotted-disclosure
When the SurfaceSelector lands:
- This type becomes one variant in the selector's typed candidate
union (alongside RefusalCandidate, TeachingCandidate, OOVCandidate).
- pack_grounded_surface() becomes a provider that emits the
candidate; the selector picks across providers' candidates by
ranked authority + is_fluent_sentence preference.
- No data migration — only the rendering step relocates.
Surface composition (chat/pack_grounding.py):
build_pack_surface_candidate(lemma) -> PackSurfaceCandidate
1. resolve_lemma(lemma) — required (None when OOV).
2. resolve_gloss(lemma) — when present AND same pack as lexicon,
compose POS-framed fluent sentence:
"Truth is a claim or state grounded by evidence and
coherent judgment. Pack-grounded (en_core_cognition_v1)."
sets is_fluent_sentence=True.
3. Else fallback to original ADR-0048 dotted-disclosure form:
"truth — pack-grounded (en_core_cognition_v1):
cognition.truth; logos.core; epistemic.ground.
No session evidence yet."
sets is_fluent_sentence=False.
pack_grounded_surface(lemma) -> str | None
Renders the candidate's surface field. Returns None for OOV.
Both fluent and disclosure surfaces carry the
"pack-grounded ({pack_id})" provenance marker so existing
substring-permissive tests continue to pass through the
transition.
POS-framed sentence templates (_frame_gloss):
NOUN -> "{Lemma} is {gloss}."
VERB -> "To {lemma} means {gloss}."
ADJ -> "Something is {lemma} when it {gloss}."
ADV -> "{Lemma} indicates {gloss}."
ADP -> "{Lemma} is a relation of {gloss}."
SCONJ -> "{Lemma} introduces {gloss}."
PRON -> "{Lemma} asks for {gloss}."
AUX -> "{Lemma} expresses {gloss}."
INTJ -> "{Lemma} is uttered to {gloss}."
DET -> "{Lemma} specifies {gloss}."
NUM -> "{Lemma} is the cardinal value {gloss}."
Phase C glosses (sitting in /tmp from the 5-subagent parallel
dispatch) are authored to fit these frames exactly.
NO GLOSSES SHIP IN THIS COMMIT. This is the wiring; the content
arrives in the Phase C commit. Today every pack still emits the
original dotted-disclosure form because no glosses.jsonl exists yet
on any pack.
Verification:
97/97 affected tests green (pack grounding, resolver, glosses,
procedure surface, correction topic, meta pack).
Cognition eval byte-identical on both splits.
Live probe with no glosses: surface format identical to pre-fix.
Lands the gloss-loader scaffolding from feat/pack-glosses-wip onto
main, with every hardening item from the 2026-05-19 design review
built in from the start. No glosses ship in this commit — only the
infrastructure that will consume them safely.
Hardening items (each pinned by a test):
1. Lexicon-residency check in resolve_gloss()
chat/pack_resolver.py — resolve_gloss now requires the lemma to be
present in the same pack's lexicon.jsonl BEFORE consulting
glosses.jsonl. Without this, glosses.jsonl would become a parallel
surface-authoring channel that bypasses the lexicon's checksum
seal: someone could ship a gloss for a lemma the pack never
ratified, and the runtime would emit it as if it were pack content.
Test: TestLexiconResidencyEnforced::test_gloss_for_unratified_lemma_is_rejected
authors a gloss for ``gamma`` (a lemma not in the lexicon) and
asserts resolve_gloss returns None.
2. Dual-checksum manifest support
language_packs/schema.py — LanguagePackManifest gains an OPTIONAL
``glosses_checksum: str | None`` field. Glosses are an additive
overlay; bumping the glosses_checksum does NOT perturb the
immutable lexicon checksum.
language_packs/compiler.py — _load_pack_cached now verifies
bytes-on-disk of glosses.jsonl against the manifest's
glosses_checksum when present. Missing field on legacy packs is
back-compat (no verification, no raise). Mismatch raises
ValueError exactly like the lexicon checksum gate.
Tests:
test_matching_glosses_checksum_loads_clean — happy path
test_checksum_mismatch_raises — tampered file rejected
test_missing_glosses_checksum_is_back_compat — legacy packs OK
3. clear_resolver_cache() clears BOTH lexicon AND glosses LRU caches
Previously only cleared _pack_lexicon_for, so test fixtures that
wrote glosses.jsonl mid-process would see stale (empty) gloss data
on subsequent resolve_gloss calls.
Test: TestClearResolverCacheClearsBoth proves the issue exists
without the clear, then proves the new code fixes it.
4. Malformed JSONL lines silently skipped
A single bad line in glosses.jsonl must not break resolution for
the rest of the pack. Same defensive parsing as _pack_lexicon_for.
Entries missing required fields (lemma, gloss, or empty values)
are also skipped.
Tests:
test_malformed_line_skipped — invalid JSON between valid lines
test_entry_missing_required_field_skipped — 4 bad shapes filtered
5. Missing glosses.jsonl is back-compat
_pack_glosses_for returns an empty dict when the file is absent.
resolve_gloss returns None. No exception. All 9 currently-
ratified English packs ship with no glosses.jsonl — they must
continue to load cleanly.
Tests:
test_pack_with_no_glosses_returns_empty
test_resolve_gloss_on_lemma_without_gloss_file_returns_none
Files:
chat/pack_resolver.py
+ _pack_glosses_for (cached loader)
+ resolve_gloss (lexicon-residency-gated lookup)
* clear_resolver_cache now clears both caches
language_packs/schema.py
+ LanguagePackManifest.glosses_checksum field (optional)
language_packs/compiler.py
+ dual-checksum verification block in _load_pack_cached
+ glosses_checksum field passed through to the manifest dataclass
tests/test_pack_resolver_glosses.py
11 tests covering all five hardening items
Verification:
11/11 new tests green.
Full cognition eval byte-identical.
All currently-ratified packs continue to load without glosses.
The 2026-05-19 design review's P0 #1 finding:
> CognitiveTurnPipeline can replace a useful runtime surface with
> placeholder prose.
Evidence at core/cognition/pipeline.py:147-149 (pre-fix):
if realized_plan.surface and not gate_fired:
surface = realized_plan.surface
articulation_surface = realized_plan.surface
The override gate was JUST "non-empty + gate didn't fire". No
usefulness check. Result: a realizer output of
"Truth is defined as ..." (with <pending> rendered as ...) silently
overrode a perfectly-grounded runtime pack surface, and the runtime
audit log still held a third surface.
Fix: gate the override through ``_is_useful_surface`` from
generate/intent_bridge.py — the same predicate that already gates
the bridge's articulate_with_intent fallback path. An ungrounded
realizer surface cannot honestly override a grounded runtime
surface. When the realizer cannot produce a useful surface, we
keep the runtime answer the user sees.
Measured lift on the warmed_session_consistency lane (3 of its 4
metrics):
BEFORE AFTER
no_placeholder_rate 0.4444 → 1.0000
telemetry_consistency_rate 0.4444 → 1.0000
warm_grounding_stability 0.0000 → 0.0000 (separate bug — see below)
The two metrics that flipped to 1.00 are now CI-pinned in
tests/test_warmed_session_lane.py:
TestPipelineOverrideGateInvariants — any future weakening of the
override gate fails the suite immediately.
Cognition eval byte-identical:
public: 100 / 100 / 91.7 / 100
holdout: 100 / 100 / 83.3 / 100
KNOWN FOLLOW-UP — not in this commit:
warm_grounding_stability remains 0.0 because of a SEPARATE bug
the warmed lane surfaces:
Turn 1: "What is truth?" -> pack-grounded ("truth — pack-grounded
(en_core_cognition_v1): cognition.truth; ...")
Turn 2: "What is truth?" -> vault-grounded ("Truth infer.")
After turn 1 ingests pack content into the vault, turn 2's gate
source flips from ``empty_vault`` to ``vault``, so the runtime's
``_maybe_pack_grounded_surface`` dispatcher is bypassed entirely
and the field-walk path produces gibberish ("Truth infer.").
This is the SurfaceSelector-shaped problem from the design review:
pack-grounding should fire by intent shape and lemma residency, not
by vault gate state. Fix scope crosses runtime.py:chat() + the
vault gate logic; deferred to its own commit / design proposal
rather than absorbed here.
The warmed lane already records the metric (0.0 baseline) so when
the fix lands it shows up as a measurable lift.
Closes the gap the 2026-05-19 design review flagged:
> Some evals are too permissive to protect fluency; they accept
> fragments or ungrammatical strings.
This lane defines fluency as six DETERMINISTIC predicates over the
user-facing surface — no LLM judge, no embedding similarity, no
aesthetics. Each predicate is a testable bool.
The six predicates:
no_placeholder — no ..., <pending>, <prior>, <empty>
no_provenance_only — surface is not bare structured disclosure
complete_punctuation — ends with . / ? / ! / ;
finite_predicate_shape — at least one finite-verb token present
no_dotted_inventory — no 3+ dotted-paths joined by ;
surface_provenance_match — grounding_source agrees with surface text
Each is a regex / substring check. Subjective fluency (rhythm,
idiom, register) is deliberately out of scope — that would require
an LLM judge (doctrine violation) or human review (not CI-pinnable).
Baseline measured on current main (this commit, all v1 public cases):
cases: 15
no_placeholder_rate: 1.0000 (hard floor — pinned)
complete_punctuation_rate: 1.0000 (hard floor — pinned)
finite_predicate_shape_rate: 1.0000 (>= 0.90 — pinned)
no_provenance_only_rate: 1.0000 (varies — lift target)
no_dotted_inventory_rate: 0.3333 (varies — lift target)
surface_provenance_match_rate: 1.0000
expected_predicates_pass_rate: 1.0000 (per-case contracts hold)
The dotted-inventory rate at 33% is the exact gap the gloss feature
is designed to close. Today 10 of 15 cases emit surfaces like
doubt — pack-grounded (en_core_meta_v1):
meta.mental_state.uncertainty; meta.mental_state; cognition.epistemic.
No session evidence yet.
After glosses land:
Doubt is a mental state of uncertainty about a claim.
Pack-grounded (en_core_meta_v1).
The lane records both metrics today; thresholds are extended in the
gloss-wiring commit so the rates DROP if the lift fails to land.
Files:
evals/deterministic_fluency/contract.md
The six predicates with implementation notes and pass thresholds.
Documents which thresholds are pinned today vs. which are gloss-
landing lift targets.
evals/deterministic_fluency/public/v1/cases.jsonl
15 cases across four categories: pack_definition (10),
oov_invitation (2), cause_no_chain_unknown_domain (2),
teaching_grounded (1). Each case declares its own
``expected_predicates`` — the subset of the six it must satisfy
today; e.g. OOV cases don't assert finite_predicate_shape because
the invitation surface is intentionally explanatory.
evals/deterministic_fluency/dev/cases.jsonl
2 representative cases for fast iteration.
evals/deterministic_fluency/runner.py
Six predicate functions + framework-compliant run_lane. Returns
per-predicate rates + per-case predicate dicts so debugging a
regression is one read of case_details away.
tests/test_deterministic_fluency_lane.py
14 contract tests covering: case-set integrity, valid predicate
names, lane discovery, every predicate rate emitted, per-case
predicates dict carries every signal, the three hard invariants
(no_placeholder == 1, complete_punctuation == 1,
finite_predicate_shape >= 0.90), expected_predicates_pass_rate
== 1 (every case satisfies its own contract), lift-target
metrics are recorded for the gloss-feature substrate.
Verification: 14/14 lane tests green on current main.
Asymmetric counterpart to cold_start_grounding. Builds the
measurement substrate for the Phase B1 pipeline-override usefulness
gate. Lane is committed now (red baseline measured) so the fix is
landed against a fixed regression target.
The 2026-05-19 design review surfaced the bug this lane catches:
> pipeline overrode a runtime surface with a placeholder realizer
> surface because realized_plan.surface was non-empty, even though
> it contained '...'. The runtime audit log still held a different
> surface. This is the central fluency/design fault: the system
> can be "green" while user-facing selection, pipeline selection,
> and telemetry selection disagree.
The lane reproduces this exactly on the current main:
Surface "Soon is defined as ..." emitted on turn 2 of "What does
soon mean?" (where turn 1 grounded as pack correctly). Telemetry
recorded a different surface than the pipeline returned.
Initial red baseline (THIS commit):
no_placeholder_rate = 0.4444 (target after Phase B1: 1.00)
telemetry_consistency_rate = 0.4444 (target after Phase B1: 1.00)
warm_grounding_stability = 0.0000 (target after Phase B1: >=0.95)
Cold-start-grounding stays at 1.00 on its own metrics. The cold lane
measures routing, the warmed lane measures override discipline; they
are deliberately not the same.
Files:
evals/warmed_session_consistency/contract.md
What is measured, why, and the asymmetry with cold_start_grounding.
Documents the four binary per-turn signals (no_placeholder,
pipeline_match_telemetry, pipeline_match_walk, grounded_holds_on_warm)
and the per-case warm_grounding_stable invariant.
evals/warmed_session_consistency/public/v1/cases.jsonl
8 cases / 18 turns. Mix of:
- replay-the-same-prompt (catches override drift)
- mixed-intent sequences (catches OOV / pack interaction)
- cause-no-chain (must stay none across replays)
- what-does-x-mean (the warmed variant of the cold-start test)
evals/warmed_session_consistency/dev/cases.jsonl
2 representative cases for fast iteration.
evals/warmed_session_consistency/runner.py
Framework-compliant run_lane(cases, config=None) -> LaneReport.
Constructs ONE ChatRuntime + CognitiveTurnPipeline per case,
plays the turn sequence through them. Per-turn signals:
no_placeholder — surface free of ..., <pending>, <prior>
telemetry_match — pipeline result.surface == turn_log[-1].surface
grounding_match — actual_grounding == expected_grounding
Per-case signal:
warm_grounding_stable — every replayed prompt produces the same
grounding across turns
tests/test_warmed_session_lane.py
8 contract tests covering: case-set integrity, replay-pattern
presence, lane discovery, runner emits every required metric,
per-turn details carry all signals, and the warmed-runtime
invariant (static check that ChatRuntime is constructed
per-case, not per-turn and not module-scope).
NOT pinned in this commit (deliberate):
Threshold assertions are NOT in the test file. They will land in
Phase B1 alongside the pipeline-override usefulness gate. This
lane's role at present is to PROVIDE the regression target, not
to enforce it before the fix.
Verification: 8/8 lane tests green; the lane itself runs and emits
the red metrics documented above.
Three independent hygiene fixes named in the 2026-05-19 design review.
All small, all observable, none architectural.
1. ``RuntimeConfig`` flag drop on pack_id / frame_pack override
chat/runtime.py:306-320 used to enumerate fields by hand when
reconstructing RuntimeConfig under the pack_id / frame_pack
override path. The list stopped at ``admissibility_margin`` and
silently dropped FIVE newer flags: identity_pack, ethics_pack,
forward_graph_constraint, composed_surface, thread_anaphora.
Caller side-effect:
ChatRuntime(pack_id="x", config=RuntimeConfig(composed_surface=True))
.config.composed_surface == False # silently lost
Fix: ``dataclasses.replace(config, input_packs=..., frame_pack=...)``.
Every field on the dataclass survives by construction; future
additions never need a synchronized edit on this path.
2. Stale CAUSE / VERIFICATION docstring
tests/test_intent_classification_extensions.py described a sixth
runtime-side fix (pack_grounded_surface fallback for
CAUSE/VERIFICATION) that was considered, reverted, and the file's
own test classes pin the opposite contract. Docstring now states
the doctrine correctly: no fallback, deliberately, so the discovery
layer can log the teaching-gap signal.
3. Thin convenience wrappers: respond / achat / arespond
tests/test_achat.py and tests/test_language_pack_runtime.py
referenced these public methods since 2026-05-14, but they were
never implemented on ChatRuntime — those 12 tests had been red on
every full-lane run since the rebase. Added as thin wrappers:
respond(text) -> ChatResponse.surface
achat(text) -> async wrapper around chat()
arespond(text)-> async wrapper around respond()
The async wrappers are deliberately NOT genuinely non-blocking —
the underlying CPU-bound walk/recall/composition remains sync.
Docstrings say so explicitly. Callers needing real concurrency
should wrap in asyncio.to_thread at the call site; promoting the
wrappers to true async event-loop integration is a future change
gated by an actual concurrent caller.
Regression coverage:
tests/test_runtime_config_passthrough.py — 4 tests
- all 19 RuntimeConfig fields survive a pack_id override
- all five newer flags survive a frame_pack override
- no-override path preserves caller config by identity (no rebuild)
- the four public methods exist and are callable
Verification:
44/44 affected tests green (was 12 red pre-fix).
Cognition eval byte-identical on both splits.
No surface-format change; this commit is pure plumbing.
Commits the 2026-05-19 probe as a durable, replayable eval lane.
This is *step 1* of the gloss-feature rollout sequence agreed
upstream: establish a stable measurement substrate before any
further intent/grounding changes, so the 52%→0% lift (and any
future regression) is reproducible and CI-pinned.
The lane is deliberately named ``cold_start_grounding`` rather than
``fluency``:
- It measures **routing** (intent → grounding source), not
sentence quality, morphology, or surface diversity.
- The cold-start qualifier reflects the fresh-``ChatRuntime()``-
per-case design. Re-using a runtime across cases would
contaminate the vault from earlier turns and was the exact bug
observed during the probe before the per-case-runtime fix.
Files:
evals/cold_start_grounding/contract.md
Lane contract: what is measured, scoring rubric, pass thresholds
(intent ≥ 0.95 / grounding ≥ 0.95 / subject ≥ 0.90), and the
rationale for the deliberate non-fallback on CAUSE/VERIFICATION
without teaching chains.
evals/cold_start_grounding/public/v1/cases.jsonl
44 cases across 16 categories. Each case carries id, prompt,
category, expected_intent, expected_grounding_source, and an
optional expected_subject. Categories cover every intent
pattern fixed in b52e04a (Define, What-does-X-mean, infinitive,
How-does-X-work, What-causes-X) plus OOV controls and CAUSE
cases with/without teaching chains.
evals/cold_start_grounding/dev/cases.jsonl
5 representative cases for fast local iteration.
evals/cold_start_grounding/runner.py
Framework-compliant ``run_lane(cases, config=None) -> LaneReport``.
Constructs a fresh ChatRuntime() inside ``_run_case`` (cold-start
invariant). Emits intent_accuracy, grounding_accuracy,
subject_accuracy, full grounding distributions, and a per-
category breakdown for regression attribution.
tests/test_cold_start_grounding_lane.py
16 contract tests covering: case-set integrity, valid enum
values, unique ids, lane discovery, pass thresholds, expected-
vs-actual distribution match (drift detection), the architectural
invariants on oov_control and cause_no_teaching_chain cases, the
cold-start invariant (static check that the runner constructs
ChatRuntime() inside the per-case helper, not at module scope),
and result JSON-serialization round-trip.
Baseline metrics (this commit, all v1 public cases):
intent_accuracy: 1.0000 (44/44)
grounding_accuracy: 1.0000 (44/44)
subject_accuracy: 1.0000 (44/44)
grounding distribution (actual == expected exactly):
pack: 37
oov: 4
teaching: 1
none: 2 (deliberate — CAUSE without teaching chain)
Why "none" cases are *expected* to ground as none:
CAUSE / VERIFICATION on a pack-resident lemma WITHOUT an active
teaching chain stays grounding_source='none' on purpose. Falling
through to pack_grounded_surface here would mask the discovery-
candidate signal the teaching pipeline uses to identify chains
worth authoring. The contract test in
TestArchitecturalInvariants::test_cause_no_chain_cases_route_to_none
pins this doctrine.
Verification: 16/16 lane tests green; full lane run via
``core eval cold_start_grounding`` reports 100% on every metric.
Subsequent steps in the agreed sequence (NOT in this commit):
2. Hygiene: runtime API wrappers (achat/arespond/respond) + the
stale CAUSE/VERIFICATION docstring in
tests/test_intent_classification_extensions.py.
3. Harden gloss resolver in feat/pack-glosses-wip
(lexicon-residency check, dual checksum, cache clearing,
malformed-JSONL skip tests).
4. Wire gloss-backed pack_grounded_surface().
5. Author starter glosses with checksum discipline.
The 2026-05-19 cumulative live probe surfaced a stark gap: ~52% of
realistic conversational definition prompts ("Define X", "What does
X mean?", "What is to V?", "How does X work?", "What causes X?")
returned ``grounding_source="none"`` *even though every subject
lemma was pack-resident* across the 9 mounted English packs.
Root cause: the bottleneck was intent classification + subject
extraction, not lexicon coverage. Five patterns either had no rule
or routed to an intent the runtime dispatcher couldn't handle. The
fluency assessment at
``/Users/kaizenpro/.codex/worktrees/6533/core/notes/fluency_assessment_2026-05-19.md``
named these as Root Cause #1 ("public chat path does not use the
cognitive spine") and Root Cause #3 ("proposition graphs are too
thin"). This commit closes the surface-level half of that gap;
the deeper answer-plan layer (gloss propositions, P3 in the
assessment) is the next step.
Patterns fixed in ``generate/intent.py``:
1. ``Define X`` — added ``^define\s+`` rule mapping to
DEFINITION (placed after ``^what is/are``
so multi-word DEFINITION patterns still
prefer the question form).
2. ``What does X mean?`` — was matching TRANSITIVE_QUERY with
relation=``mean``. Now re-routes to
DEFINITION inside ``classify_intent`` so
``pack_grounded_surface`` fires on X.
Other transitive relations (precede,
ground, etc.) remain TRANSITIVE_QUERY.
3. ``What is to V?`` — added infinitive-marker strip to
``_normalize_subject`` for DEFINITION /
RECALL. ``to`` is gated on intent tag so
it never strips a transfer preposition
from CAUSE / VERIFICATION.
4. ``How does X work?`` — added ``_HOW_DOES_X_RE`` (third-person
mechanistic-cause). Distinct from the
first-person PROCEDURE rule ("How do I
X?"). Verbs: work / function / operate /
happen / exist / behave / act / emerge.
5. ``What causes X?`` — added causative-verb rule (causes /
triggers / enables / prevents / drives /
produces / induces / yields) routing to
CAUSE with X as subject.
Deliberate NON-fix: I considered adding a ``pack_grounded_surface``
fallback in the CAUSE / VERIFICATION dispatcher when no teaching
chain matches the subject. Reverted on review — that masks the
"would_have_grounded" discovery-candidate signal the teaching
pipeline uses to identify teaching-content gaps (see
``tests/test_discovery_candidates``). CAUSE on a pack-resident
lemma without a teaching chain stays ``grounding_source=='none'``
so the discovery layer can log the gap honestly.
``chat/pack_grounding.py``:
Extended ``_CORRECTION_TOPIC_STOPWORDS`` to include polarity
markers (no / yes / maybe / perhaps / hardly / indeed / surely /
definitely). Without this the CORRECTION composer would
short-circuit on ``no`` from "No, my parent disagrees" and miss
the topical lemma ``parent``.
Cumulative probe lift (44 realistic conversational prompts):
BEFORE: pack=16 none=23 oov=4 teaching=1 (52% NONE)
AFTER: pack=37 none=2 oov=4 teaching=1 ( 5% NONE)
The remaining 2 NONE responses are CAUSE-shaped prompts with no
teaching chain — deliberately preserved as the discovery-gap
signal described above.
Tests: tests/test_intent_classification_extensions.py — 23 new
tests covering each pattern + the lift invariant.
Verification:
Cognition eval byte-identical on both splits (100/100/91.7/100
public, 100/100/83.3/100 holdout).
All 111 intent-affected tests green:
test_intent_classification_extensions.py (23)
test_intent_proposition_graph.py / test_intent_ratifier.py /
test_intent_subject_extraction.py / test_narrative_example_intents.py
test_procedure_surface.py
test_correction_topic_lemma.py
test_cross_pack_grounding.py (including the polarity-stopword fix)
test_discovery_candidates.py
test_contemplation_wiring.py
test_en_core_polarity_v1_pack.py
Workstream 1 eighth pack. Closes the polarity-marker + frequency-
adverb gap. Common conversational markers (yes/no/maybe/always/never)
had zero coverage in any prior pack.
Pack composition (16 entries — 2 INTJ / 14 ADV):
polarity.affirm.* yes indeed surely definitely
polarity.negate.* no hardly
polarity.uncertain.* maybe perhaps
polarity.frequency.* always sometimes often rarely never
usually occasionally frequently
``certain``/``certainly``/``uncertain`` deliberately excluded — those
remain in en_core_attitude_v1 (epistemic.certainty/uncertainty).
Regression test pins the invariant.
tests/test_correction_topic_lemma.py:
Three fixtures swapped from "No that is wrong" to "Nope that is
wrong". ``no`` is now correctly pack-resident in en_core_polarity_v1
(polarity.negate.dissent), so the "no pack-resident lemma" contract
these tests pin needed a fixture where every content token is
genuinely OOV. ``nope`` is OOV across all 10 mounted packs; ``wrong``
remains OOV (collision with attitude's ``right`` blocked spatial-
direction ``right`` but did not add ``wrong``).
Authoring:
Three parallel subagents — affirm / negate+uncertain / frequency.
Workstream 1 sixth pack. Closes the spatial-vocabulary gap. Prior
packs had zero coverage of here/there, location nouns, or spatial
prepositions.
Pack composition (24 entries — 7 ADV / 8 ADP / 9 NOUN):
spatial.deictic.* here there (2 ADV)
spatial.direction.* forward backward left up down (5 ADV)
spatial.relation.* near far above below inside outside
between beyond (8 ADP)
spatial.noun.* place location area region space
end top bottom side (9 NOUN)
``right`` was deliberately omitted — en_core_attitude_v1 already owns
it as evaluative.positive, and first-match-wins resolution preserves
that claim. A regression test pins this invariant explicitly.
Files: lexicon.jsonl / manifest.json + 12 contract tests.
Verification: full lane 2204 passed / 2 skipped / 0 failed.
Cognition eval byte-identical both splits.
Workstream 1 fifth pack. Closes the quantifier + basic-numeric gap.
Prior packs had zero coverage of universal / existential / comparative
quantifiers — queries about *all*, *some*, *many*, *more*, *most* all
fell through to OOV.
Pack composition (24 entries — mixed POS, 18 DET / 3 NUM / 2 ADJ / 1 NOUN):
quantitative.universal.* (6 DET) all every each both none neither
quantitative.existential.* (6 DET) some any several few many much
quantitative.comparative.* (6 DET) more less fewer most least enough
quantitative.numeric.* (3 NUM) one two three
quantitative.unit.* (3 mix) single (ADJ) half (NOUN) whole (ADJ)
The composer is POS-agnostic; surface composition uses
``semantic_domains`` rather than POS, so DET/NUM/ADJ/NOUN entries all
surface identically.
Files:
language_packs/data/en_core_quantitative_v1/
lexicon.jsonl — 24 entries, SHA-256 checksum-sealed
manifest.json — operational_base / D0
chat/pack_resolver.py
Appended to DEFAULT_RESOLVABLE_PACK_IDS after action.
core/config.py
Added to RuntimeConfig.input_packs default mount.
tests/test_en_core_quantitative_v1_pack.py
11 contract tests (load / POS-dist / namespace / no-collision /
contiguous-ids / mount / resolver-order / routing / invariance).
Authoring:
Three parallel subagents — universal+existential / comparative /
numeric. Strict exemplar + forbidden-lemma list against all 7
prior packs.
Verification:
Full lane: 2192 passed, 2 skipped, 0 failed.
Cognition eval byte-identical on both splits.
Workstream 1 fourth pack. Closes the common-action verb gap. Prior
packs covered reasoning (cognition), speech/perception (meta), and
adjectives (attitude); this pack covers what an agent *does*.
Pack composition (26 VERB entries):
action.doing.perform do perform execute carry conduct
action.doing.make make
action.doing.achieve achieve accomplish
action.creating.originate create build form produce generate develop
action.changing.transform change transform
action.moving.translate move
action.moving.depart_arrive go come
action.moving.transfer send receive
action.possessing.acquire get take
action.possessing.transfer give
action.possessing.retain keep
action.possessing.deploy use
Files:
language_packs/data/en_core_action_v1/
lexicon.jsonl — 26 entries, SHA-256 checksum-sealed
manifest.json — operational_base / D0
chat/pack_resolver.py
Appended to DEFAULT_RESOLVABLE_PACK_IDS after temporal.
core/config.py
Added to RuntimeConfig.input_packs default mount.
tests/test_en_core_action_v1_pack.py
11 contract tests covering load / POS / namespace / no-collision /
contiguous-ids / mounted-by-default / resolver-order / routing /
prior-pack invariance.
tests/test_procedure_surface.py
Swapped two test fixtures from "do stuff" to "fix bugs". ``do``
is now correctly pack-resident in en_core_action_v1 (semantically
correct — "How do I do stuff?" should ground on ``do``), so the
"no pack lemma exists" contract needed a fixture where both verb
and noun are genuinely OOV. ``fix bugs`` satisfies this across
all 7 mounted packs.
Authoring:
Three parallel subagents — doing / creating / moving+possessing.
Strict exemplar + forbidden-lemma list against all 6 prior packs.
Verification:
Cognition eval byte-identical on both splits (100/100/91.7/100 and
100/100/83.3/100).
All 70 pack tests pass (cognition + meta + attitude + temporal +
action + quant tests run together).
Live composer probes confirm every action lemma surfaces
deterministically from en_core_action_v1.
Workstream 1 third pack. Closes the temporal-vocabulary gap — prior
to this pack zero time/sequence/aspect terms existed in any mounted
English pack, so queries about *when*, *before*, *after*, *now*,
*future*, *past* all fell through to OOV.
Pack composition (28 entries, mixed POS — 12 ADV / 9 NOUN / 5 ADP /
1 SCONJ / 1 ADJ):
temporal.deictic.* (10 ADV) now today tomorrow yesterday soon
later recently eventually currently
formerly
temporal.relative.* (9 mix) before after during while until since
ago prior henceforth
temporal.noun.* (9 NOUN) moment period duration instant era
future past present time
The pack composer is POS-agnostic — surface composition uses the
ratified ``semantic_domains`` list rather than the POS tag. Mixed-POS
entries surface identically to noun/verb entries.
Files:
language_packs/data/en_core_temporal_v1/
lexicon.jsonl — 28 entries, SHA-256 checksum-sealed
manifest.json — operational_base / D0 / checksum-verified
chat/pack_resolver.py
Appended to DEFAULT_RESOLVABLE_PACK_IDS after attitude.
core/config.py
Added to RuntimeConfig.input_packs default mount.
tests/test_en_core_temporal_v1_pack.py
11 contract tests: checksum, POS-distribution invariant, primary-
domain namespace, no-collision regression gate against all 5 prior
packs, contiguous entry_ids, mounted-by-default, resolver-order
invariant, routing correctness, and prior-pack resolution unchanged.
Authoring:
Three parallel subagents — deictic / relative / nouns. Strict
exemplar + forbidden-lemma list against all 5 prior packs.
Verification:
Full lane: 2170 passed, 2 skipped, 0 failed (+11 new tests).
Cognition eval byte-identical on both splits.
Live composer probes confirm every temporal lemma surfaces
deterministically from en_core_temporal_v1.
Workstream 1 second pack. Closes the ADJ POS gap — prior to this pack
zero adjectives existed in any mounted English content pack, so the
runtime could not emit grounded surfaces for predicative queries like
"What is true?" or "What is important?".
Pack composition (40 ADJ entries):
attitude.truth_value.* (8) true false valid invalid accurate
inaccurate factual sound
attitude.evaluative.* (6) good bad right better worse best
attitude.epistemic.* (10) certain uncertain possible impossible
likely unlikely probable clear obscure
evident
attitude.modal.* (4) necessary sufficient required optional
attitude.importance.* (6) important essential relevant central
primary useful
attitude.scope.* (6) general specific broad narrow universal
particular
Files:
language_packs/data/en_core_attitude_v1/
lexicon.jsonl — 40 entries, SHA-256 checksum-sealed
manifest.json — operational_base / D0 / checksum-verified
chat/pack_resolver.py
Appended to DEFAULT_RESOLVABLE_PACK_IDS after cognition + meta.
core/config.py
Added to RuntimeConfig.input_packs default mount.
tests/test_en_core_attitude_v1_pack.py
11 contract tests: checksum, POS=ADJ uniformity, primary-domain
namespace, no-collision regression gate against all 4 prior packs,
contiguous entry_ids, mounted-by-default, resolver-order invariant,
routing correctness, and cognition+meta resolution unchanged.
Authoring:
Three parallel subagents (1 per cluster) — truth/eval, epistemic/modal,
importance/scope. Strict exemplar + forbidden-lemma list against all
prior packs. Main pass assembled, validated, sealed.
Verification:
Full lane: 2159 passed, 2 skipped, 0 failed (+11 new tests over the
previous 2148 baseline).
Cognition eval byte-identical on both splits:
public 100 / 100 / 91.7 / 100
holdout 100 / 100 / 83.3 / 100
Live composer probes: every ADJ lemma emits a deterministic
pack-grounded surface from en_core_attitude_v1.
Workstream 1 (pack content scale-up) first load-bearing step.
Adds a new ratified content pack covering the conversational vocabulary
en_core_cognition_v1 deliberately omits — speech acts, mental states,
perception, self-reference, and discourse-object nouns. These are the
lemmas that show up in nearly every model response and that previously
fell through to the OOV invitation surface.
Pack composition (73 entries, 49 VERB + 24 NOUN):
meta.speech_act.* (20 verbs) say tell speak reply claim state
describe express name mention note
observe declare assert deny confirm
suggest propose articulate respond
meta.mental_state.* (18 verbs) know believe think suppose assume
expect hope want prefer doubt wonder
guess recognize realize consider intend
decide hold
meta.perception.* (11 verbs) see hear feel sense perceive watch
look listen find detect notice
meta.self_reference.* (10 nouns) self mind view perspective position
role agent model system speaker
meta.discourse.* (14 nouns) response reply statement fact idea
point argument proposal suggestion
case instance example kind type
Files:
language_packs/data/en_core_meta_v1/
lexicon.jsonl — 73 entries, SHA-256 checksum-sealed
manifest.json — operational_base / D0 / checksum-verified
chat/pack_resolver.py
Appended en_core_meta_v1 to DEFAULT_RESOLVABLE_PACK_IDS after
en_core_cognition_v1 so cognition lemma resolution stays first-
match-wins on any future collision (preserves cognition-lane
byte-identity invariant).
core/config.py
Added en_core_meta_v1 to RuntimeConfig.input_packs default mount.
tests/test_en_core_meta_v1_pack.py
11 contract tests: checksum-verified load, POS split, primary-
domain namespace, no-collision-with-cognition-v1 regression gate,
pack registration order, resolver routing, and cognition-lemma
resolution unchanged.
tests/test_procedure_surface.py
Swapped two test fixtures from "claim" to "hypothesis". ``claim``
is now correctly pack-resident (meta.speech_act.claim) so the
procedure composer's object-first selector picks it over the verb
— the new behavior is semantically correct. ``hypothesis`` is
genuinely OOV across all mounted packs and preserves the verb-
fallback contract these tests pin.
Authoring methodology:
Four parallel subagents authored one cluster each from a strict
exemplar + word list + forbidden-lemma list (every en_core_cognition_v1
lemma listed explicitly to prevent collision). Each subagent wrote
only its cluster JSONL; the main pass assembled, validated, computed
the SHA-256 over bytes-on-disk, and wrote the manifest.
Verification:
Full lane: 2148 passed, 2 skipped, 0 failed (+11 new tests).
Cognition eval byte-identical on both splits:
public 100 / 100 / 91.7 / 100
holdout 100 / 100 / 83.3 / 100
Live runtime probes: fresh ChatRuntime() for "What is X?" with
X ∈ {fact, doubt, statement, model, self} all emit a
pack-grounded sentence from en_core_meta_v1.
OOV path still honest for genuinely-unknown terms (e.g. hypothesis).
Scope note:
This is one pack of ~70 lemmas, not "the model now articulates
open-domain English." The architecturally-honest articulation
story still requires more pack and teaching-chain content; this
pack moves the conversational-substrate boundary forward by ~70
lemmas in one ratifiable, replay-stable step.
Root cause: recalled_words was built from result.tokens (versor walk
neighbours) rather than the pack-resolved proposition slots. The walk
produces nearest-neighbour traversal artifacts; the proposition already
carries the correct subject/predicate/object from realize(). This made
ground_graph() fill <pending> obj slots with stop-word-adjacent tokens
instead of the actual answer content.
Fix — two changes, one new helper:
generate/intent_bridge.py
• build_recalled_words_from_plan(plan, proposition, walk_tokens)
Constructs the grounding tuple in priority order:
1. plan.object (ArticulationPlan — pack-resolved, already a word)
2. proposition.object_ (Proposition — versor-decoded object slot)
3. plan.predicate (descriptive predicate word, richer than walk)
4. plan.subject (subject as last-resort semantic anchor)
5. walk_tokens (result.tokens alpha-filtered — supplemental backfill)
Strips <pending>/<prior>/empty/non-alpha before deduplicating.
Returns a deduplicated tuple in that priority order.
• articulate_with_intent() gains an optional `proposition` param
(typed as object to avoid import coupling at the call site).
When provided, build_recalled_words_from_plan() is called to
replace the raw recalled_words before ground_graph() runs.
When omitted, behaviour is byte-identical to Phase 1 (backward
compatible: all existing callers and tests pass unchanged).
chat/runtime.py
• The single articulate_with_intent() call site now passes
proposition=proposition so the bridge receives the full
pack-resolved proposition for grounding. walk_tokens (the old
recalled_words) are passed through as supplemental backfill.
• No change to ChatResponse, TurnEvent, GenerationResult, or any
ADR-gated schema.
Adds generate/bridge_trace.py: a structured sink + serializer for
per-turn articulation-bridge trace records, following the exact
ADR-0040 telemetry sink pattern (JsonlBufferSink / JsonlFileSink /
FanOutSink, no wall-clock, redact-by-default).
Modifies generate/intent_bridge.py: articulate_with_intent() emits
one BridgeTraceRecord per call through a module-level opt-in sink
(attach_bridge_trace_sink / detach_bridge_trace_sink). When no
sink is attached the call is a pure no-op — zero behavior change on
all existing paths.
The record captures:
- intent_tag / intent_subject (classifier output)
- plan_subject / plan_predicate / plan_object (articulation slots)
- recalled_words_len / recalled_words_sample (grounding supply)
- pre_ground_obj (what the graph node held before ground_graph)
- post_ground_obj (what it held after, or same if no grounding ran)
- bridge_surface / bridge_useful (final output + usefulness gate)
- fallback_surface (the plan.surface the runtime falls back to)
This is the Phase 1 measurement instrumentation described in the
full-sentence output mastery plan. Phases 2-5 act on the data this
produces; Phase 1 itself is pure observation.
Phase 5 (ADR-0067 follow-up):
teaching/cross_pack_supersede.py — supersede_cross_pack_chain()
CLI: core teaching supersede ... --cross-pack
--subject-pack-id ... --object-pack-id ...
Strict per-chain residency, anti-leakage, byte-identical rollback
on any post-append re-load failure. 9 new tests.
Articulation benchmark suite (Phase 4 capability proof):
benchmarks/articulation.py — 5 sub-benches
[1] breadth — every intent shape (9 + OOV + cross-pack)
[2] determinism — N reruns / unique-surface count
[3] footprint — psutil RSS profile across T turns
[4] cross-topic — thread context across mixed subjects
[5] ollama-compare — opt-in side-by-side with local Ollama
CLI: core bench --suite articulation
--runs N (det rerun count)
--turns N (footprint sample window)
--ollama-model MODEL --ollama-reruns N
Full operator preamble + JSON report path.
10 new tests cover the bench shape (psutil import-skipped).
Documentation:
benchmarks/README.md — full operator manual: catalogue of every
bench suite, how to read good/neutral/bad results for each sub-
bench, why CORE vs Ollama comparisons are valid on the
determinism axis and not on linguistic quality, workflow guide.
README.md — articulation bench listed in the live-demo grid and
quick-start examples.
Reference run (llama3:8b, 100 turns, 5 reruns):
determinism_all_identical=True
per-turn ΔRSS ≈ 23 KiB
CORE byte_identical_on_every_prompt=True
Ollama unique_surfaces≥2 on every prompt
Verification:
18 new tests pass
Full lane: 2116 passed, 2 skipped, 0 failed in 2:38
Two new intent shapes + composers turn the runtime's corpus
density into operator-visible articulation. Both consult the
cross-corpus aggregator from ADR-0064; no new ratification needed.
P3.3 — chat/narrative_surface.py + IntentTag.NARRATIVE.
Classifier patterns (registered BEFORE generic DEFINITION):
^tell\s+me\s+about\s+
^describe\s+
^what\s+(?:can|do)\s+you\s+(?:say|know)\s+about\s+
narrative_grounded_surface(subject, max_clauses=4) walks every
reviewed chain rooted on subject across all registered teaching
corpora. Dedupes by (connective, object) — cause + verification
carrying the same predicate emit one clause, not two. Sorts by
(intent, connective, object) for replay stability.
Surface format:
"{X} — narrative-grounded ({corpus_ids}): {dX1}; {dX2}.
{X} {conn1} {O1} ({dO1}); {X} {conn2} {O2} ({dO2}).
No session evidence yet."
Cross-corpus subjects (e.g. mother in relations_v2) emit
narrative-grounded (relations_chains_v2) tag; cognition subjects
emit cognition_chains_v1 tag. Multi-corpus subjects (when
applicable) emit composite "corpus_a + corpus_b" tag.
P3.4 — chat/example_surface.py + IntentTag.EXAMPLE.
Classifier patterns:
^(?:give|show)\s+(?:me\s+)?an?\s+(?:example|instance)\s+of\s+
^example\s+of\s+
example_grounded_surface(object_lemma, max_examples=3) walks chains
where the lemma is the OBJECT — inverts the typical subject-keyed
access pattern. Dedupes by subject; sorts by (intent, subject,
connective).
Surface format:
"{X} — example-grounded ({corpus_ids}): {dX1}.
Example: {subj1} {conn1} {X}; {subj2} {conn2} {X}.
No session evidence yet."
Cross-cutting:
- Both intents added to _OOV_INTENT_TAGS — fall through to OOV
invitation when subject is unknown (Phase 2 gradient discipline).
- Both tagged grounding_source="teaching" (same provenance tier
as the existing teaching_grounded_surface).
- No prose generation, no new mutation surface.
Live verification:
> Tell me about truth.
[teaching] truth — narrative-grounded (cognition_chains_v1):
cognition.truth; logos.core. truth grounds knowledge
(cognition.knowledge); truth requires evidence (cognition.evidence).
> Give me an example of knowledge.
[teaching] knowledge — example-grounded (cognition_chains_v1):
cognition.knowledge. Example: truth grounds knowledge;
understanding requires knowledge; evidence grounds knowledge.
> Tell me about mother.
[teaching] mother — narrative-grounded (relations_chains_v2):
kinship.parent.female. mother precedes daughter (kinship.child.female).
> Describe photosynthesis.
[oov] I haven't learned 'photosynthesis' yet (intent: narrative). ...
ADR-0066 (this commit completes the ADR). 30 new tests passed.
Full lane: 2067 passed, 2 skipped, 0 failed in 2:32.
Shay