Phase C of the gloss feature. Lands the natural-language gloss
content that the resolver (Phase B2) and the runtime composer
(Phase B3) were prepared for. This is the user-visible payoff:
cold-start DEFINITION / RECALL prompts on pack-resident lemmas now
emit fluent grounded sentences instead of dotted-domain disclosure.
Authoring: five parallel subagents in ONE message block (a single
parallel dispatch, ~20s wall-clock vs ~95s sequential). Each
subagent received its pack's complete lemma + POS list and a strict
JSON-shape exemplar. Total returned: 326 raw gloss entries.
Assembly (this commit): the raw entries were partitioned by
lexicon-residency lookup (the resolve_gloss invariant enforced at
storage time), deduplicated within pack, sorted by lemma, written
to ``language_packs/data/<pack>/glosses.jsonl``, and each pack's
manifest received a new ``glosses_checksum`` field. 323 glosses
landed clean; 0 rejected.
Per-pack distribution:
en_core_cognition_v1 78 glosses
en_core_meta_v1 72 glosses
en_core_attitude_v1 40 glosses
en_core_temporal_v1 28 glosses
en_core_action_v1 26 glosses
en_core_quantitative_v1 24 glosses
en_core_spatial_v1 24 glosses
en_core_polarity_v1 16 glosses
en_core_causation_v1 15 glosses
Live-probe lift (fresh ChatRuntime per prompt):
BEFORE:
truth — pack-grounded (en_core_cognition_v1):
cognition.truth; logos.core; epistemic.ground.
No session evidence yet.
AFTER:
Truth is a claim or state grounded by evidence and coherent
judgment. pack-grounded (en_core_cognition_v1).
Same provenance. Same audit-trail content (the dotted domains are
still in lexicon.jsonl, the resolver can still read them, the
candidate object carries them verbatim). But the user-facing
surface is a sentence the user can actually read.
Eval-lane lift:
deterministic_fluency BEFORE AFTER
no_dotted_inventory_rate 0.3333 → 1.0000
no_provenance_only_rate 1.0000 → 1.0000 (held)
no_placeholder_rate 1.0000 → 1.0000 (held)
complete_punctuation_rate 1.0000 → 1.0000 (held)
finite_predicate_shape 1.0000 → 1.0000 (held)
surface_provenance_match 1.0000 → 1.0000 (held)
cold_start_grounding all metrics held at 1.0
warmed_session_consistency no_placeholder + telemetry_match held at 1.0
(warm_grounding_stability still 0 — separate fix)
cognition eval public 100 / 100 / 91.7 / 100 (BYTE-IDENTICAL)
cognition eval holdout 100 / 100 / 83.3 / 100 (BYTE-IDENTICAL)
The cognition eval bytes-identity holds because the eval checks
substring containment (case-insensitive after the format change).
Every lemma still appears in its fluent surface.
Hardening this commit enforces:
Lexicon-residency at storage time
tests/test_pack_glosses_content.py::test_every_gloss_lemma_is_lexicon_resident
walks every glosses.jsonl and asserts every lemma is present in
the same pack's lexicon.jsonl. Drift in glosses (an unratified
lemma sneaking in) fails the lane immediately.
Dual-checksum discipline
tests/test_pack_glosses_content.py::test_every_glossed_pack_has_matching_checksum
re-hashes glosses.jsonl bytes-on-disk and compares against the
manifest's glosses_checksum. Any tampering fails.
Immutable-lexicon invariant
tests/test_pack_glosses_content.py::test_lexicon_checksum_unchanged_by_gloss_landing
re-hashes lexicon.jsonl and compares against the manifest's
(original) checksum. Proves that adding glosses did NOT perturb
the lexicon seal.
High-freq lemma resolution
32 of the most-common conversational lemmas (truth, doubt,
fact, idea, self, true, important, now, place, make, effect,
always, ...) all resolve to a fluent surface end-to-end.
Test-suite drift this commit absorbed:
- tests/test_pack_grounding.py — three substring assertions
updated to be case-insensitive (gloss-backed surfaces capitalize
lemmas at sentence start, dotted-disclosure surfaces don't).
"No session evidence yet" assertion replaced with the
common-substring "pack-grounded" marker that BOTH forms emit.
- tests/test_pack_resolver_glosses.py — the back-compat test
pivots from en_core_cognition_v1 (now glossed) to en_minimal_v1
(deliberately unglossed). A new test pins the glossed case.
Files added:
language_packs/data/<pack>/glosses.jsonl (9 files, 323 entries)
tests/test_pack_glosses_content.py (9 contract tests)
Files modified:
language_packs/data/<pack>/manifest.json (9 files, glosses_checksum field)
chat/pack_grounding.py (lowercase "pack-grounded" tag)
tests/test_pack_grounding.py (3 substring assertions relaxed)
tests/test_pack_resolver_glosses.py (back-compat test pivoted)
Verification:
127/127 affected tests green.
9/9 new gloss-content tests green.
All three eval lanes report the lift documented above.
Cognition eval byte-identical.
Replaces the flat "I don't know — insufficient grounding" disclosure
with a deterministic gradient that names specific vocabulary gaps
and gives operators concrete next steps.
P2.1 — OOV "teach me" surface (chat/oov_surface.py).
When the intent classifier extracts a clean subject lemma but that
lemma is not resident in any mounted lexicon pack, the runtime now
emits a deterministic learning-invitation surface tagged
``grounding_source="oov"`` instead of the universal disclosure.
Surface format (fixed template):
"I haven't learned '{token}' yet (intent: {intent}).
Mounted lexicon packs: {pack_list}.
Teach me via a reviewed PackMutationProposal."
The OOV token passes through ``core._safe_display.safe_display``
before persistence — user-input sanitization at the trust boundary.
No vocabulary is invented; no domain is inferred. Honours the
ADR-0027 proposal-only invariant: the surface invites a reviewed
pack mutation, never silently mutates any pack.
Refactored ``_maybe_pack_grounded_surface`` so every existing
intent branch (COMPARISON / CAUSE / VERIFICATION / CORRECTION /
PROCEDURE / DEFINITION+RECALL) falls through on a None composer
result instead of early-returning. The OOV invitation is the
deterministic fall-through for any clean-subject prompt whose
subject doesn't resolve.
P2.2 — Partial-grounding tier (chat/partial_surface.py).
When exactly one of two COMPARISON lemmas resolves, the runtime
emits a hedged surface that grounds the known side verbatim and
disclaims the OOV side explicitly:
"Whatever '{oov}' is, I can ground '{known}' — pack-grounded
({pack_id}): {d1}; {d2}. I cannot ground the comparison
without learning '{oov}' — teach me via a reviewed
PackMutationProposal."
Tagged ``grounding_source="partial"``. Falls through to OOV
invitation when both lemmas are OOV, and to full pack-grounded
COMPARISON when both resolve — partial is the middle tier in the
five-tier gradient.
Also normalises trailing sentence punctuation on
intent.secondary_subject at the COMPARISON boundary so prompts
like "Compare A and B." (with the period) still resolve B
correctly.
Five-tier gradient (vault → teaching → pack → partial → oov → none).
Test debt retired: four pre-existing tests asserted "OOV → universal
disclosure", which is exactly the contract P2.1/P2.2 inverted.
Rewritten to the new contract. Plus test_procedure_surface.py
gained a test for the OOV gradient on procedure intents.
Verification:
tests/test_oov_surface.py 22 passed
tests/test_partial_surface.py 16 passed
Cognition eval byte-identical:
public 100% / 100% / 91.7% / 100%
holdout 100% / 100% / 83.3% / 100%
Curated lanes all green.
ADR-0063 closes the ADR-0048/0050/0053/0061 hardcoded-cognition-pack
asymmetry. New chat/pack_resolver.py provides resolve_lemma(lemma,
pack_ids) → (resolving_pack_id, semantic_domains) across an ordered
tuple of mounted lexicon packs (first-match-wins, lru_cache per-pack).
Surface composers in chat/pack_grounding.py now consult the resolver
instead of a hardcoded en_core_cognition_v1. en_core_relations_v1
joins RuntimeConfig.input_packs defaults; kinship lemmas now ground
on the live path:
> What is a parent?
parent — pack-grounded (en_core_relations_v1):
kinship.ascendant.direct; kinship.parent; biology.progenitor.
No session evidence yet.
Cross-pack comparison (knowledge × parent) renders composite tag
(en_core_cognition_v1 × en_core_relations_v1). Cognition lane
remains byte-identical: cognition is resolved first and the surface
format for cognition lemmas is unchanged.
Cognition eval (byte-identical to pre-ADR baseline):
public → intent 100% / surface 100% / term 91.7% / closure 100%
holdout → intent 100% / surface 100% / term 83.3% / closure 100%
Curated lanes green: smoke 67 / cognition 121 / teaching 17 /
packs 6 / runtime 19 / algebra 132.
New tests: test_pack_resolver.py (28) + test_cross_pack_grounding.py
(17). test_en_core_relations_v1_pack.py: default-input-packs guard
inverted. test_pack_grounding.py: two stale ADR-0048 tests rewritten
(premises invalidated by ADR-0052/0061; now use fully-out-of-pack
prompts).
chat/teaching_grounding.py UNCHANGED — cognition_chains_v1 corpus
stays cognition-only. Cross-pack teaching corpora are the natural
ADR-0064.
Closes the surface-grounding gap isolated by ADR-0047's
characterisation. Adds the ratified cognition pack as a second
grounding source alongside the session vault.
== chat/pack_grounding.py (new) ==
Loads en_core_cognition_v1's lexicon once (cached; immutable pack)
and exposes:
pack_grounded_surface(lemma) -> str | None
Returns a deterministic, fully pack-sourced surface:
"{lemma} — pack-grounded ({pack_id}): {d1}; {d2}; {d3}. No session evidence yet."
Every visible atom is the lemma or a verbatim semantic_domains
string from the pack. No rewording, no synthesis, no LLM.
== chat/runtime.py ==
_stub_response gains optional pack_grounded_surface= parameter.
_maybe_pack_grounded_surface routes to the pack only when all four
hold: gate_source=="empty_vault", output_language=="en",
intent.tag in {DEFINITION, RECALL}, and intent.subject is a pack
lemma. Safety/ethics refusal still takes priority above this branch.
ChatResponse and TurnEvent gain grounding_source ∈ {vault,pack,none}.
Main walk path tags responses "vault".
== core/cognition/pipeline.py ==
gate_fired detection moved from string equality on the universal
disclosure to provenance:
gate_fired = response.vault_hits == 0 and response.grounding_source != "vault"
Same intent (suppress realizer template on gate-fired turns),
broader stub-path surface set.
== Characterisation (core eval cognition, 13-case public split) ==
Metric Pre Post Δ
intent_accuracy 100.0% 100.0% 0
surface_groundedness 15.4% 46.2% +30.8 pp
term_capture_rate 0.0% 33.3% +33.3 pp
versor_closure_rate 100.0% 100.0% 0
Lift is non-uniform by design: only single-lemma DEFINITION/RECALL
on pack-known English subjects engage. CAUSE/COMPARISON/VERIFICATION
and multi-word OOV subjects still return the universal disclosure —
fabricating those would violate the no-LLM-fallback doctrine.
== Tests ==
tests/test_pack_grounding.py 18 passed
tests/test_semantic_realizer_integration.py (updated) 1 stub-path test
pinned to the broader contract: surface is either universal
disclosure or pack-grounded; never the realizer template.
== Lanes ==
smoke 67 cognition 121 runtime 19 algebra 132
teaching 17 packs 6
versor_condition(F) < 1e-6 invariant unaffected (no algebra changes).
Shay