Commit graph

4 commits

Author SHA1 Message Date
Shay
cb1eba72ae feat(packs): en_core_action_v1 — action verbs (26 lemmas)
Workstream 1 fourth pack.  Closes the common-action verb gap.  Prior
packs covered reasoning (cognition), speech/perception (meta), and
adjectives (attitude); this pack covers what an agent *does*.

Pack composition (26 VERB entries):

  action.doing.perform     do perform execute carry conduct
  action.doing.make        make
  action.doing.achieve     achieve accomplish
  action.creating.originate create build form produce generate develop
  action.changing.transform change transform
  action.moving.translate  move
  action.moving.depart_arrive go come
  action.moving.transfer   send receive
  action.possessing.acquire get take
  action.possessing.transfer give
  action.possessing.retain keep
  action.possessing.deploy use

Files:
  language_packs/data/en_core_action_v1/
    lexicon.jsonl   — 26 entries, SHA-256 checksum-sealed
    manifest.json   — operational_base / D0
  chat/pack_resolver.py
    Appended to DEFAULT_RESOLVABLE_PACK_IDS after temporal.
  core/config.py
    Added to RuntimeConfig.input_packs default mount.
  tests/test_en_core_action_v1_pack.py
    11 contract tests covering load / POS / namespace / no-collision /
    contiguous-ids / mounted-by-default / resolver-order / routing /
    prior-pack invariance.
  tests/test_procedure_surface.py
    Swapped two test fixtures from "do stuff" to "fix bugs".  ``do``
    is now correctly pack-resident in en_core_action_v1 (semantically
    correct — "How do I do stuff?" should ground on ``do``), so the
    "no pack lemma exists" contract needed a fixture where both verb
    and noun are genuinely OOV.  ``fix bugs`` satisfies this across
    all 7 mounted packs.

Authoring:
  Three parallel subagents — doing / creating / moving+possessing.
  Strict exemplar + forbidden-lemma list against all 6 prior packs.

Verification:
  Cognition eval byte-identical on both splits (100/100/91.7/100 and
  100/100/83.3/100).
  All 70 pack tests pass (cognition + meta + attitude + temporal +
  action + quant tests run together).
  Live composer probes confirm every action lemma surfaces
  deterministically from en_core_action_v1.
2026-05-19 05:38:12 -07:00
Shay
a376a30bf8 feat(packs): en_core_meta_v1 — conversational substrate (73 lemmas)
Workstream 1 (pack content scale-up) first load-bearing step.

Adds a new ratified content pack covering the conversational vocabulary
en_core_cognition_v1 deliberately omits — speech acts, mental states,
perception, self-reference, and discourse-object nouns.  These are the
lemmas that show up in nearly every model response and that previously
fell through to the OOV invitation surface.

Pack composition (73 entries, 49 VERB + 24 NOUN):

  meta.speech_act.*     (20 verbs)  say tell speak reply claim state
                                    describe express name mention note
                                    observe declare assert deny confirm
                                    suggest propose articulate respond
  meta.mental_state.*   (18 verbs)  know believe think suppose assume
                                    expect hope want prefer doubt wonder
                                    guess recognize realize consider intend
                                    decide hold
  meta.perception.*     (11 verbs)  see hear feel sense perceive watch
                                    look listen find detect notice
  meta.self_reference.* (10 nouns)  self mind view perspective position
                                    role agent model system speaker
  meta.discourse.*      (14 nouns)  response reply statement fact idea
                                    point argument proposal suggestion
                                    case instance example kind type

Files:
  language_packs/data/en_core_meta_v1/
    lexicon.jsonl   — 73 entries, SHA-256 checksum-sealed
    manifest.json   — operational_base / D0 / checksum-verified
  chat/pack_resolver.py
    Appended en_core_meta_v1 to DEFAULT_RESOLVABLE_PACK_IDS after
    en_core_cognition_v1 so cognition lemma resolution stays first-
    match-wins on any future collision (preserves cognition-lane
    byte-identity invariant).
  core/config.py
    Added en_core_meta_v1 to RuntimeConfig.input_packs default mount.
  tests/test_en_core_meta_v1_pack.py
    11 contract tests: checksum-verified load, POS split, primary-
    domain namespace, no-collision-with-cognition-v1 regression gate,
    pack registration order, resolver routing, and cognition-lemma
    resolution unchanged.
  tests/test_procedure_surface.py
    Swapped two test fixtures from "claim" to "hypothesis".  ``claim``
    is now correctly pack-resident (meta.speech_act.claim) so the
    procedure composer's object-first selector picks it over the verb
    — the new behavior is semantically correct.  ``hypothesis`` is
    genuinely OOV across all mounted packs and preserves the verb-
    fallback contract these tests pin.

Authoring methodology:
  Four parallel subagents authored one cluster each from a strict
  exemplar + word list + forbidden-lemma list (every en_core_cognition_v1
  lemma listed explicitly to prevent collision).  Each subagent wrote
  only its cluster JSONL; the main pass assembled, validated, computed
  the SHA-256 over bytes-on-disk, and wrote the manifest.

Verification:
  Full lane: 2148 passed, 2 skipped, 0 failed (+11 new tests).
  Cognition eval byte-identical on both splits:
    public  100 / 100 / 91.7 / 100
    holdout 100 / 100 / 83.3 / 100
  Live runtime probes: fresh ChatRuntime() for "What is X?" with
  X ∈ {fact, doubt, statement, model, self} all emit a
  pack-grounded sentence from en_core_meta_v1.
  OOV path still honest for genuinely-unknown terms (e.g. hypothesis).

Scope note:
  This is one pack of ~70 lemmas, not "the model now articulates
  open-domain English."  The architecturally-honest articulation
  story still requires more pack and teaching-chain content; this
  pack moves the conversational-substrate boundary forward by ~70
  lemmas in one ratifiable, replay-stable step.
2026-05-19 05:38:12 -07:00
Shay
51aad0c2cd feat(adr-0065): OOV cliff → five-tier honesty gradient (Phase 2.1 + 2.2)
Replaces the flat "I don't know — insufficient grounding" disclosure
with a deterministic gradient that names specific vocabulary gaps
and gives operators concrete next steps.

P2.1 — OOV "teach me" surface (chat/oov_surface.py).

  When the intent classifier extracts a clean subject lemma but that
  lemma is not resident in any mounted lexicon pack, the runtime now
  emits a deterministic learning-invitation surface tagged
  ``grounding_source="oov"`` instead of the universal disclosure.

  Surface format (fixed template):

    "I haven't learned '{token}' yet (intent: {intent}).
     Mounted lexicon packs: {pack_list}.
     Teach me via a reviewed PackMutationProposal."

  The OOV token passes through ``core._safe_display.safe_display``
  before persistence — user-input sanitization at the trust boundary.
  No vocabulary is invented; no domain is inferred.  Honours the
  ADR-0027 proposal-only invariant: the surface invites a reviewed
  pack mutation, never silently mutates any pack.

  Refactored ``_maybe_pack_grounded_surface`` so every existing
  intent branch (COMPARISON / CAUSE / VERIFICATION / CORRECTION /
  PROCEDURE / DEFINITION+RECALL) falls through on a None composer
  result instead of early-returning.  The OOV invitation is the
  deterministic fall-through for any clean-subject prompt whose
  subject doesn't resolve.

P2.2 — Partial-grounding tier (chat/partial_surface.py).

  When exactly one of two COMPARISON lemmas resolves, the runtime
  emits a hedged surface that grounds the known side verbatim and
  disclaims the OOV side explicitly:

    "Whatever '{oov}' is, I can ground '{known}' — pack-grounded
     ({pack_id}): {d1}; {d2}.  I cannot ground the comparison
     without learning '{oov}' — teach me via a reviewed
     PackMutationProposal."

  Tagged ``grounding_source="partial"``.  Falls through to OOV
  invitation when both lemmas are OOV, and to full pack-grounded
  COMPARISON when both resolve — partial is the middle tier in the
  five-tier gradient.

  Also normalises trailing sentence punctuation on
  intent.secondary_subject at the COMPARISON boundary so prompts
  like "Compare A and B." (with the period) still resolve B
  correctly.

Five-tier gradient (vault → teaching → pack → partial → oov → none).

Test debt retired: four pre-existing tests asserted "OOV → universal
disclosure", which is exactly the contract P2.1/P2.2 inverted.
Rewritten to the new contract.  Plus test_procedure_surface.py
gained a test for the OOV gradient on procedure intents.

Verification:
  tests/test_oov_surface.py                       22 passed
  tests/test_partial_surface.py                   16 passed
  Cognition eval byte-identical:
    public  100% / 100% / 91.7% / 100%
    holdout 100% / 100% / 83.3% / 100%
  Curated lanes all green.
2026-05-18 16:41:45 -07:00
Shay
bf7f7895fe feat(adr-0061): PROCEDURE intent routes to pack-grounded surface
Pre-ADR-0061 every "How do I X?" question fell through to the
universal disclosure even when X was a pack-resident lemma.  The
teaching corpus carries CAUSE/VERIFICATION chains only — procedural
knowledge is fundamentally different in kind from propositional
claims and deserves its own ratification path (deliberately out of
scope; a future parallel `procedure_chains_v1.jsonl` schema is
discussed in the ADR's non-goals).

ADR-0061 adds the honest cold-start fallback: ground the topic in
pack semantic_domains and note explicitly that ratified step-by-step
guidance does not exist yet.

Surface format:
  "procedure-grounded ({pack_id}): {lemma} ({d1}; {d2}).
   Step-by-step guidance for {lemma} is not yet ratified
   in this session."

Selector — **last** pack-resident lemma in the verb-phrase subject:
  "define a concept" → concept    (object beats verb)
  "verify a claim"   → verify     (verb wins when object is OOV)
  "correct an error" → correct
  "learn this"       → learn
  "do stuff"         → None       (falls through to universal disclosure)

Stopwords: only `be` and `have` (dialogue fillers).  Procedure verbs
are deliberately NOT stopworded so the verb-as-fallback rule fires
when the object is OOV — keeps surface coverage.

Trust-boundary invariants:
  - Every visible non-template token is lemma / pack-domain / template.
  - Deterministic: same subject_text → same bytes.
  - Returns None for fully-unknown utterances → universal disclosure
    fires.  Never fabricates surface from nothing (ADR-0053 contract).
  - "not yet ratified" trust-label preserved.

Cognition lane lift:
  public  : intent 100% / surface 100% / term 91.7% / versor 100%      (unchanged)
  holdout : intent 100% / surface 94.7%→100.0% / term 79.2%→83.3% / versor 100%

Two cases fixed:
  - procedure_define_010 ("How do I define a concept?") — surface +
    term `concept` now captured.
  - procedure_verify_034 ("How do I verify a claim?") — surface only
    (case has no expected_terms; the verb fallback grounds it).

Combined effect: holdout `surface_groundedness` closes to 100%; 4 of
5 architectural holdout misses now resolved (this ADR + ADR-0060 +
the supersede from epistemology v1).  Remaining 2 are UNKNOWN-intent
cases (unknown_spirit_041, unknown_word_018) — out of scope; deserve
their own ADR with distinct selector semantics.

- chat/pack_grounding.py — `_extract_procedure_topic_lemma` helper +
  `pack_grounded_procedure_surface` composer.
- chat/runtime.py — import + dispatch branch for `IntentTag.PROCEDURE`.
- tests/test_procedure_surface.py — 15 tests pin: extraction
  (last-wins / verb-by-elimination / be+have skipped / None on empty /
  strips punctuation / case-insensitive); surface (contains lemma /
  contains domains / pack_id / "not yet ratified" label / None for
  no-pack-lemma / deterministic); end-to-end through ChatRuntime.

Lanes (regression): smoke 67 / cognition 121 / teaching 17 /
procedure 15 — all green.

The non-negotiable field invariant (versor_condition < 1e-6) is
unaffected: this ADR changes surface composition only.
2026-05-18 14:22:19 -07:00