core/docs/handoff/ADR-0084-pack-content-brief.md
Shay 48282eef8d
feat(adr-0084): definitional layer — proposal + substrate (schema/loader/closure) (#64)
* docs(adr-0084): propose definitional layer + prompt-diversity suite

Three companion artifacts proposing the next substantive design step
after ADR-0083:

1. ADR-0084 (Proposed) — Definitional Layer for Lexicon Packs
   Optional `definition` block on pack entries: gloss,
   definitional_atoms, predicates_invited, definition_version,
   provenance.  Pack-level opt-in.  Closure rule: every word in a
   gloss must resolve to a same-pack lemma, another mounted pack's
   lemma, or a primitive in a new `packs/primitives/` pack.
   NO composer change in this ADR (sequenced for ADR-0085) —
   ratify substrate before any consumer depends on it.

2. evals/prompt_diversity/ (Proposed) — companion eval lane
   ~50 cases across question-shape × sophistication × domain,
   measuring three new metrics: response_shape_fit,
   audit_in_surface_rate (quantifies the trust-boundary leak into
   user surfaces), gloss_quote_rate (zero today; rises with future
   gloss-aware composer).  No v1 pass thresholds — the lane
   establishes a baseline distribution so future work has
   something to move.  26 seed cases authored covering all 21
   categories.

3. docs/handoff/ADR-0084-pack-content-brief.md — paste-ready brief
   for a cheaper/faster dev agent to produce the pack content in
   parallel.  Self-contained, 5 sequenced phases (primitives pack
   → extend 9 existing glosses → add to relations/anchors → write
   closure verifier → run safety lanes), explicit don't-touch list
   (no composer / runtime / algebra / Greek+Hebrew packs / schema
   parser), no-LLM-glosses discipline, per-phase acceptance.

Discovery while drafting: 9 packs already carry glosses.jsonl
under language_packs/data/ with a flat schema (78 entries in
en_core_cognition_v1 alone).  The brief reflects that — most
work is extending existing entries, not authoring from scratch.

Strategic context: ADR-0083 raised the *depth* ceiling on chain
composition; ADR-0084 raises the *fidelity* ceiling.  The φ
separation probe (memory: phi-separation-falsified) established
that semantic capability lives in chain composition, not in φ
geometry, so deepening the composer's substrate is the natural
next step.  ADR-0084 → 0085 (gloss-aware composer) → 0086
(predicate licensing at ratification) is the planned sequence.

* feat(adr-0084): substrate — schema parser, primitives loader, closure verifier

Substrate-only code-side for ADR-0084 (Definitional Layer for Lexicon Packs).
No composer touches the new fields yet; consumer integration is ADR-0085.

Schema (additive, default preserves byte-identity)
  - LanguagePackManifest.definitional_layer: bool = False
  - compiler loader propagates the flag from manifest.json

language_packs/definitions.py (new)
  - GlossEntry dataclass: lemma, gloss, pos, definitional_atoms,
    predicates_invited, definition_version, provenance_ids
  - parse_gloss_entry(payload, *, strict) — strict mode enforces ADR-0084
    §Schema validation row-by-row: required keys, typed lists, no
    unknown keys, positive definition_version; lax mode preserves the
    legacy two-field shape for back-compat
  - load_pack_glosses(pack_id, *, strict) with cache + clear hook
  - verify_definitional_closure(pack_id, *, mounted_pack_lemmas,
    primitive_lemmas, strict) returning tuple[ClosureViolation, ...];
    case-insensitive resolution; cycles permitted per ADR

packs/primitives/loader.py (new)
  - Sister loader to packs/safety/ and packs/identity/
  - PrimitivesPack frozen dataclass with .lemmas frozenset
  - Gates: checksum match, kind=='primitives', definitional_layer:true,
    never_auto_mutable:true, pack_id matches dir, primitive_count
    cross-check, duplicate-lemma rejection, path-traversal rejection,
    strict per-entry schema with allow-list
  - DEFAULT_PRIMITIVES_PACK = 'en_semantic_primitives_v1'

tests/test_adr_0084_definitional_substrate.py
  - 38 tests covering strict parser (each required key rejection, unknown
    key rejection, empty predicates_invited allowed, empty
    definitional_atoms rejected, invalid definition_version), lax
    parser back-compat, load_pack_glosses (missing/strict raise/lax
    skip/malformed JSON), closure verifier (same-pack/primitive/mounted/
    unresolved/case-insensitive), primitives loader (every gate), and
    a back-compat check that every shipped pack still ratifies with
    definitional_layer=False

Lanes: smoke 67/0, cognition 120/0/1, teaching 17/0, runtime 19/0,
packs 6/0. Cognition eval byte-identical 100/91.7/100/100.

When the content PR lands (primitives.jsonl + extended glosses.jsonl
under ADR-0084-pack-content-brief.md), the gate catches any closure-rule
violation without further code change.

* feat(evals): prompt_diversity lane runner — measurement instrument for ADR-0084+

Implements the runner against the existing contract.md + 26-case v1
public split.  Lane auto-discovered by evals.framework via the standard
contract + runner convention.

Runner (evals/prompt_diversity/runner.py)
  - run_lane(cases, *, config, workers) -> LaneReport
  - 5 metrics: intent_accuracy, versor_closure_rate (carried over from
    cognition), plus the three new lane-specific metrics —
    response_shape_fit, audit_in_surface_rate, gloss_quote_rate
  - breakdown dict groups by (question_shape, sophistication, domain)
    per contract §How to read the output
  - mirrors evals.cognition.runner's parallel worker pattern

Per-shape classifier (deliberately substring/regex-simple at v1)
  - predicate_identity, explanation, sequence, two_subject_contrast,
    narrative, honest_disclosure
  - Unknown shape => neutral pass (don't penalise new categories)

Audit-leak detector
  - trust-boundary preamble markers (teaching-grounded (, pack-grounded
    (, No session evidence yet.)
  - dotted semantic-domain tag regex (cognition.illumination, etc.)

Gloss-quote detector
  - resolves expected_terms via chat.pack_resolver.resolve_gloss
  - 4-token contiguous-window match against surface (high-confidence
    "gloss actually quoted", not "shared one common word")

Tests (tests/test_prompt_diversity_runner.py — 23)
  - shape classifier parametrized over the six expected_shape values
  - audit-leak detector parametrized over preamble + tag + clean cases
  - end-to-end on v1 public:
      * versor_closure_rate == 1.0 (only v1 pass threshold per contract)
      * every metric in [0, 1]
      * breakdown groups present with the four per-cell metrics
      * diversity gate: >=5 question shapes, >=3 domains
        (defends against future regressions that collapse the suite
         back to a cognition-shaped fixture)

v1/public baseline (26 cases)
  intent_accuracy      : 65.4%   (contract predicted 70-85%)
  versor_closure_rate  : 100.0%  (only v1 pass threshold)  PASS
  response_shape_fit   : 53.8%   (contract predicted low)
  audit_in_surface_rate: 42.3%   (contract predicted ~100%)
  gloss_quote_rate     :  7.7%   (contract predicted 0%)

Three baseline surprises worth noting in the report (NOT failures —
the v1 lane is explicitly there to establish the distribution):

  - audit_in_surface_rate at 42% (not 100%) means the chain-walk leak
    fires on ~11/26; the other 15 are honest-disclosure cases that
    emit no audit envelope.  Sharpens the future surface-vs-envelope
    ADR's actual target: grounded surfaces specifically.
  - response_shape_fit at 54% (not "low") — classifier likely has
    false positives on the ", which " cause-marker.  Worth tightening
    once we have an ADR-0085 baseline to compare against.
  - intent_accuracy at 65% (below predicted 70-85%) — classifier dips
    harder on adversarial/cross-pack than expected.  Real gap.

All five smoke/cognition/teaching/runtime/packs lanes still green;
core eval cognition byte-identical 100/91.7/100/100.

* feat(packs): ADR-0084 pack content (primitives + extend glosses + closure verifier) (#65)

* feat(packs): ADR-0084 pack content

* feat(packs): repair ADR-0084 definitional content

* test(adr-0084): adjust substrate manifest tests for post-#65 content reality

PR #65 flipped definitional_layer:true on 13 English packs (9 core +
4 relations + collapse-anchors).  The substrate's previous test
test_existing_packs_unchanged asserted that en_core_cognition_v1 +
en_core_relations_v1 still had definitional_layer:False — which was
the right pre-content invariant but is wrong post-content.

Replace it with two complementary tests that hold against real content:

  - test_non_opted_packs_default_false:
      pins that packs that DIDN'T flip the flag (en_minimal_v1,
      he_core_cognition_v1, grc_logos_cognition_v1) still surface
      definitional_layer=False through the loader.  Defends against
      a future change accidentally flipping the flag on a non-opted
      pack.

  - test_opted_packs_carry_flag:
      pins that packs that DID flip the flag (en_core_cognition_v1,
      en_core_relations_v1) surface definitional_layer=True through
      the loader.  Proves the substrate's manifest-field propagation
      works against real ratified content, not just fixture packs.

Net: +1 test, same intent (substrate ratifies the manifest field
correctly), now with real-content coverage on both sides of the gate.

All 62 ADR-0084 substrate + prompt-diversity tests pass.
2026-05-20 15:25:25 -07:00

12 KiB
Raw Blame History

ADR-0084 Pack-Content Build Brief

Audience: A fresh dev agent (cheaper/faster tier). You have NO prior context — read this file and docs/decisions/ADR-0084-definitional-layer.md and act from there. Do not invent design decisions; if something is unclear, surface a question rather than guessing.

Mission: Produce the content required by ADR-0084 (Definitional Layer for Lexicon Packs). You are not implementing the schema parser, the composer integration, or any runtime code. You are only producing JSONL pack files and verifying they cohere.


What already exists (do not duplicate)

language_packs/data/
├── en_core_cognition_v1/      ← HAS glosses.jsonl (78 entries)
├── en_core_meta_v1/           ← HAS glosses.jsonl (72)
├── en_core_attitude_v1/       ← HAS glosses.jsonl (40)
├── en_core_polarity_v1/       ← HAS glosses.jsonl (16)
├── en_core_action_v1/         ← HAS glosses.jsonl (26)
├── en_core_temporal_v1/       ← HAS glosses.jsonl (28)
├── en_core_spatial_v1/        ← HAS glosses.jsonl (24)
├── en_core_causation_v1/      ← HAS glosses.jsonl (15)
├── en_core_quantitative_v1/   ← HAS glosses.jsonl (24)
├── en_core_relations_v1/      ← lexicon only, no glosses
├── en_core_relations_v2/      ← lexicon only, no glosses
├── en_core_relations_v3/      ← lexicon only, no glosses
├── en_collapse_anchors_v1/    ← lexicon only, no glosses
├── he_core_cognition_v1/      ← lexicon only, no glosses
├── grc_logos_cognition_v1/    ← lexicon only, no glosses
├── grc_logos_micro_v1/        ← lexicon only, no glosses
├── he_logos_micro_v1/         ← lexicon only, no glosses
└── en_minimal_v1/             ← lexicon only, no glosses (skip — staging)

Existing glosses.jsonl entries have the flat shape:

{"lemma":"analogy","gloss":"a mapping of relations shared between distinct domains","pos":"NOUN","provenance_ids":["seed:glosses_v1"]}

They DO NOT yet carry definitional_atoms or predicates_invited — those fields are what ADR-0084 introduces. You will extend each existing entry with those two fields, in place, preserving every other field byte-identically.


Phase 1 — Primitives pack (highest priority, blocks everything else)

Create packs/primitives/en_semantic_primitives_v1/. This is a new top-level directory; create it if it does not exist.

Files to create

  1. packs/primitives/en_semantic_primitives_v1/manifest.json
  2. packs/primitives/en_semantic_primitives_v1/primitives.jsonl
  3. packs/primitives/en_semantic_primitives_v1/README.md (one paragraph)

Primitives discipline

A primitive is a word whose meaning is taken as terminal at the system level — it does not require a gloss because every other definition can bottom out on it. Pick ~4060 primitives across these tight categories:

Category Examples
Existence exist, be, not_be
Identity same, different, this, that
Relation relation, cause, part, whole, connected_to
Mode way, means, medium
Disclosure visible, hidden, seen, known
Quantity (basic) one, many, none, some
State change, still, start, end
Reference who, what, where, when, why, how
Communication say, mean, ask, answer
Logic if, then, and, or, because

Avoid overlap with the existing 9 core packs — primitives must be the floor, not duplicates of vocabulary already glossed elsewhere. If you see a word in any existing glosses.jsonl, prefer to leave it out of primitives.

Primitive entry schema

{
  "lemma": "exist",
  "category": "existence",
  "pos": "VERB",
  "primitive_version": 1,
  "provenance_ids": ["adr-0084:reviewed:2026-05-20"]
}

No gloss. No definitional_atoms. Primitives are terminal.

Manifest

{
  "pack_id": "en_semantic_primitives_v1",
  "language": "en",
  "kind": "primitives",
  "definitional_layer": true,
  "version": 1,
  "issued_at": "2026-05-20T00:00:00Z",
  "checksum": "<sha256 of primitives.jsonl as written>",
  "primitive_count": <N>,
  "never_auto_mutable": true,
  "provenance": "adr-0084:reviewed:2026-05-20"
}

Compute checksum as hashlib.sha256(Path("primitives.jsonl").read_bytes()).hexdigest() AFTER writing the file (the manifest hashes the bytes actually on disk).

Acceptance for Phase 1

  • File exists at the path above.
  • 40 ≤ primitive_count ≤ 60.
  • Every line of primitives.jsonl is valid JSON with the schema above.
  • No primitive lemma appears in any existing glosses.jsonl (run a quick grep -l "\"lemma\":\"<word>\"" language_packs/data/*/glosses.jsonl for each — should return empty).
  • Manifest checksum matches sha256 of primitives.jsonl bytes.

Phase 2 — Extend existing 9 core glosses to ADR-0084 schema

For each pack in language_packs/data/ that already has a glosses.jsonl, extend each entry to:

{
  "lemma": "analogy",
  "gloss": "a mapping of relations shared between distinct domains",
  "pos": "NOUN",
  "definitional_atoms": ["mapping", "relation", "shared", "distinct", "domain"],
  "predicates_invited": ["reveals", "illustrates", "maps"],
  "definition_version": 1,
  "provenance_ids": ["seed:glosses_v1", "adr-0084:reviewed:2026-05-20"]
}

definitional_atoms rule

Every content word in gloss (skip articles, prepositions, "is", "of", "to", "by", "the", "a", "an", "that", "which") must be in definitional_atoms. Each entry in definitional_atoms must resolve to EXACTLY ONE of:

  1. Another lemma in the SAME pack's lexicon.jsonl or glosses.jsonl.
  2. A lemma in another pack's lexicon.jsonl or glosses.jsonl under language_packs/data/.
  3. A primitive in packs/primitives/en_semantic_primitives_v1/primitives.jsonl.

If a word in the gloss does not resolve, do ONE of:

  • Rephrase the gloss using a word that does resolve (preferred).
  • Add the unresolved word to the primitives pack (only if it is genuinely terminal — surface a note in your final report listing every word you promoted to primitive).
  • Add a new gloss to the most appropriate existing core pack (e.g. if "mapping" belongs in en_core_action_v1, add it there with its own closure).

NEVER invent a new pack just to hold one stray word.

predicates_invited rule

List 26 predicates that this lemma legitimately appears with as the subject. Examples:

  • light["reveals", "illuminates", "shines", "exposes"]
  • knowledge["requires", "grounds", "supports", "informs"]
  • parent["raises", "begets", "guides", "shelters"]

If you are unsure for a given lemma, set predicates_invited: [] (the v1 spec accepts empty during migration). Prefer empty over guessing.

Acceptance for Phase 2

  • Every pre-existing glosses.jsonl entry gains definitional_atoms, predicates_invited, definition_version, and an additional adr-0084:reviewed:2026-05-20 provenance entry.
  • Run the closure check (Phase 4) and fix any unresolved references before moving on.
  • Each pack's manifest.json is updated:
    • Add "definitional_layer": true.
    • Refresh checksum against the new glosses.jsonl bytes.
    • Append adr-0084 to provenance.

Phase 3 — New glosses for packs that lack them

Add glosses.jsonl to each of these packs, one entry per lemma in the pack's lexicon.jsonl:

  • en_core_relations_v1, v2, v3 (kinship — parent, child, sibling, family, ancestor, descendant, spouse, offspring, and the v2/v3 additions)
  • en_collapse_anchors_v1 (only 3 lemmas — small)

Use the same schema as Phase 2. Same closure rule. Same predicates_invited discipline.

DO NOT touch these in this brief:

  • he_core_cognition_v1, grc_logos_cognition_v1, grc_logos_micro_v1, he_logos_micro_v1 — Greek/Hebrew per-lens glosses are a separate ADR (anchor-lens substrate, deferred per ADR-0084 scope limits).
  • en_minimal_v1 — staging pack, not production.

Acceptance for Phase 3

  • Each named pack has a glosses.jsonl with one entry per lexicon.jsonl entry.
  • All Phase 2 acceptance items apply.

Phase 4 — Closure verification

Write a stand-alone Python script at scripts/verify_definitional_closure.py that:

  1. Loads every glosses.jsonl under language_packs/data/ whose pack manifest has "definitional_layer": true.
  2. Loads packs/primitives/en_semantic_primitives_v1/primitives.jsonl.
  3. For each gloss entry, for each token in definitional_atoms, checks it resolves to ONE of (a) same-pack lemma, (b) other-pack lemma, (c) primitive.
  4. Prints a report:
    • total entries checked
    • total definitional_atoms tokens checked
    • unresolved tokens grouped by pack
    • exit code 0 iff zero unresolved
  5. Optional --json flag for machine-readable output.

This script is the ratification gate proxy until the real one ships in the schema parser ADR. Run it after every batch of edits.

Acceptance for Phase 4

  • Script exists, is runnable as python -m scripts.verify_definitional_closure.
  • Exit code 0 on the final pack state.
  • A --json invocation produces a parseable report.

Phase 5 — Run the safety lanes

After every phase, run these and confirm green:

PYTHONPATH=. python3 -m core.cli test --suite smoke -q
PYTHONPATH=. python3 -m core.cli test --suite packs -q
PYTHONPATH=. python3 -m core.cli test --suite cognition -q
PYTHONPATH=. python3 -m core.cli eval cognition

If any of these breaks, STOP and surface the failure. Do not patch around it — the composer was not changed by this ADR, so any cognition/teaching regression means a pack content drift broke an existing invariant. The fix is to the pack content, not to the runtime.

Acceptance for Phase 5

  • All four green.
  • core eval cognition metrics byte-identical to pre-brief baseline (capture the baseline FIRST — run the eval before starting Phase 1 and save the output for diff).

Things you must NOT do

  • DO NOT modify any file under chat/, core/, generate/, vault/, field/, algebra/, teaching/, evals/ (except for reading them).
  • DO NOT modify language_packs/compiler.py or language_packs/data/<pack>/manifest.json fields other than checksum, provenance, and the new definitional_layer flag.
  • DO NOT introduce any LLM-generated glosses by routing through an external API. Every gloss is hand-written, drawn from primitives + co-pack vocabulary. If a lemma is genuinely outside your domain knowledge, surface it as a question instead of guessing.
  • DO NOT touch Greek / Hebrew packs (per ADR scope limit).
  • DO NOT add predicates_invited you cannot defend — empty is the correct default for uncertainty.
  • DO NOT skip checksum refresh — packs are content-addressed; stale checksums fail ratification.

Reporting

When you are done (or blocked), produce a single markdown report:

# ADR-0084 Pack-Content Build — Final Report

## Phase 1 — primitives pack
- created: <path>
- primitive_count: <N>
- categories covered: <list>
- words promoted to primitive (during Phase 2 closure): <list>

## Phase 2 — existing glosses extended
- packs touched: <list>
- entries extended: <total>
- entries left with predicates_invited=[]: <count>
- closure status: <verified | issues>

## Phase 3 — new glosses created
- packs touched: <list>
- entries added: <total>

## Phase 4 — closure verifier
- script path: <path>
- final exit code: <0 | nonzero>

## Phase 5 — safety lanes
- smoke: <pass | fail>
- packs: <pass | fail>
- cognition: <pass | fail>
- eval cognition metrics diff: <byte-identical | drift details>

## Open questions / handoff items
- <list anything you couldn't answer without guessing>

Surface this report at the end. Do not commit. The human reviews the report before any merge.


Sanity primitives (a starter list — extend, don't blindly copy)

You may use these as a starting point for the primitives pack. Confirm each is NOT already in an existing glosses.jsonl before including:

exist, be, not_be, same, different, this, that, relation, cause, part,
whole, connected_to, way, means, medium, visible, hidden, seen, known,
one, many, none, some, change, still, start, end, who, what, where,
when, why, how, say, mean, ask, answer, if, then, and, or, because,
through, from, into, between, before, after

That's 49. Decide which deserve to be in the primitives pack and which are better placed inside an existing core pack (e.g. through, from, into, between, before, after are likely already in en_core_spatial_v1 or en_core_temporal_v1).

End of brief.