* docs(adr-0084): propose definitional layer + prompt-diversity suite
Three companion artifacts proposing the next substantive design step
after ADR-0083:
1. ADR-0084 (Proposed) — Definitional Layer for Lexicon Packs
Optional `definition` block on pack entries: gloss,
definitional_atoms, predicates_invited, definition_version,
provenance. Pack-level opt-in. Closure rule: every word in a
gloss must resolve to a same-pack lemma, another mounted pack's
lemma, or a primitive in a new `packs/primitives/` pack.
NO composer change in this ADR (sequenced for ADR-0085) —
ratify substrate before any consumer depends on it.
2. evals/prompt_diversity/ (Proposed) — companion eval lane
~50 cases across question-shape × sophistication × domain,
measuring three new metrics: response_shape_fit,
audit_in_surface_rate (quantifies the trust-boundary leak into
user surfaces), gloss_quote_rate (zero today; rises with future
gloss-aware composer). No v1 pass thresholds — the lane
establishes a baseline distribution so future work has
something to move. 26 seed cases authored covering all 21
categories.
3. docs/handoff/ADR-0084-pack-content-brief.md — paste-ready brief
for a cheaper/faster dev agent to produce the pack content in
parallel. Self-contained, 5 sequenced phases (primitives pack
→ extend 9 existing glosses → add to relations/anchors → write
closure verifier → run safety lanes), explicit don't-touch list
(no composer / runtime / algebra / Greek+Hebrew packs / schema
parser), no-LLM-glosses discipline, per-phase acceptance.
Discovery while drafting: 9 packs already carry glosses.jsonl
under language_packs/data/ with a flat schema (78 entries in
en_core_cognition_v1 alone). The brief reflects that — most
work is extending existing entries, not authoring from scratch.
Strategic context: ADR-0083 raised the *depth* ceiling on chain
composition; ADR-0084 raises the *fidelity* ceiling. The φ
separation probe (memory: phi-separation-falsified) established
that semantic capability lives in chain composition, not in φ
geometry, so deepening the composer's substrate is the natural
next step. ADR-0084 → 0085 (gloss-aware composer) → 0086
(predicate licensing at ratification) is the planned sequence.
* feat(adr-0084): substrate — schema parser, primitives loader, closure verifier
Substrate-only code-side for ADR-0084 (Definitional Layer for Lexicon Packs).
No composer touches the new fields yet; consumer integration is ADR-0085.
Schema (additive, default preserves byte-identity)
- LanguagePackManifest.definitional_layer: bool = False
- compiler loader propagates the flag from manifest.json
language_packs/definitions.py (new)
- GlossEntry dataclass: lemma, gloss, pos, definitional_atoms,
predicates_invited, definition_version, provenance_ids
- parse_gloss_entry(payload, *, strict) — strict mode enforces ADR-0084
§Schema validation row-by-row: required keys, typed lists, no
unknown keys, positive definition_version; lax mode preserves the
legacy two-field shape for back-compat
- load_pack_glosses(pack_id, *, strict) with cache + clear hook
- verify_definitional_closure(pack_id, *, mounted_pack_lemmas,
primitive_lemmas, strict) returning tuple[ClosureViolation, ...];
case-insensitive resolution; cycles permitted per ADR
packs/primitives/loader.py (new)
- Sister loader to packs/safety/ and packs/identity/
- PrimitivesPack frozen dataclass with .lemmas frozenset
- Gates: checksum match, kind=='primitives', definitional_layer:true,
never_auto_mutable:true, pack_id matches dir, primitive_count
cross-check, duplicate-lemma rejection, path-traversal rejection,
strict per-entry schema with allow-list
- DEFAULT_PRIMITIVES_PACK = 'en_semantic_primitives_v1'
tests/test_adr_0084_definitional_substrate.py
- 38 tests covering strict parser (each required key rejection, unknown
key rejection, empty predicates_invited allowed, empty
definitional_atoms rejected, invalid definition_version), lax
parser back-compat, load_pack_glosses (missing/strict raise/lax
skip/malformed JSON), closure verifier (same-pack/primitive/mounted/
unresolved/case-insensitive), primitives loader (every gate), and
a back-compat check that every shipped pack still ratifies with
definitional_layer=False
Lanes: smoke 67/0, cognition 120/0/1, teaching 17/0, runtime 19/0,
packs 6/0. Cognition eval byte-identical 100/91.7/100/100.
When the content PR lands (primitives.jsonl + extended glosses.jsonl
under ADR-0084-pack-content-brief.md), the gate catches any closure-rule
violation without further code change.
* feat(evals): prompt_diversity lane runner — measurement instrument for ADR-0084+
Implements the runner against the existing contract.md + 26-case v1
public split. Lane auto-discovered by evals.framework via the standard
contract + runner convention.
Runner (evals/prompt_diversity/runner.py)
- run_lane(cases, *, config, workers) -> LaneReport
- 5 metrics: intent_accuracy, versor_closure_rate (carried over from
cognition), plus the three new lane-specific metrics —
response_shape_fit, audit_in_surface_rate, gloss_quote_rate
- breakdown dict groups by (question_shape, sophistication, domain)
per contract §How to read the output
- mirrors evals.cognition.runner's parallel worker pattern
Per-shape classifier (deliberately substring/regex-simple at v1)
- predicate_identity, explanation, sequence, two_subject_contrast,
narrative, honest_disclosure
- Unknown shape => neutral pass (don't penalise new categories)
Audit-leak detector
- trust-boundary preamble markers (teaching-grounded (, pack-grounded
(, No session evidence yet.)
- dotted semantic-domain tag regex (cognition.illumination, etc.)
Gloss-quote detector
- resolves expected_terms via chat.pack_resolver.resolve_gloss
- 4-token contiguous-window match against surface (high-confidence
"gloss actually quoted", not "shared one common word")
Tests (tests/test_prompt_diversity_runner.py — 23)
- shape classifier parametrized over the six expected_shape values
- audit-leak detector parametrized over preamble + tag + clean cases
- end-to-end on v1 public:
* versor_closure_rate == 1.0 (only v1 pass threshold per contract)
* every metric in [0, 1]
* breakdown groups present with the four per-cell metrics
* diversity gate: >=5 question shapes, >=3 domains
(defends against future regressions that collapse the suite
back to a cognition-shaped fixture)
v1/public baseline (26 cases)
intent_accuracy : 65.4% (contract predicted 70-85%)
versor_closure_rate : 100.0% (only v1 pass threshold) PASS
response_shape_fit : 53.8% (contract predicted low)
audit_in_surface_rate: 42.3% (contract predicted ~100%)
gloss_quote_rate : 7.7% (contract predicted 0%)
Three baseline surprises worth noting in the report (NOT failures —
the v1 lane is explicitly there to establish the distribution):
- audit_in_surface_rate at 42% (not 100%) means the chain-walk leak
fires on ~11/26; the other 15 are honest-disclosure cases that
emit no audit envelope. Sharpens the future surface-vs-envelope
ADR's actual target: grounded surfaces specifically.
- response_shape_fit at 54% (not "low") — classifier likely has
false positives on the ", which " cause-marker. Worth tightening
once we have an ADR-0085 baseline to compare against.
- intent_accuracy at 65% (below predicted 70-85%) — classifier dips
harder on adversarial/cross-pack than expected. Real gap.
All five smoke/cognition/teaching/runtime/packs lanes still green;
core eval cognition byte-identical 100/91.7/100/100.
* feat(packs): ADR-0084 pack content (primitives + extend glosses + closure verifier) (#65)
* feat(packs): ADR-0084 pack content
* feat(packs): repair ADR-0084 definitional content
* test(adr-0084): adjust substrate manifest tests for post-#65 content reality
PR #65 flipped definitional_layer:true on 13 English packs (9 core +
4 relations + collapse-anchors). The substrate's previous test
test_existing_packs_unchanged asserted that en_core_cognition_v1 +
en_core_relations_v1 still had definitional_layer:False — which was
the right pre-content invariant but is wrong post-content.
Replace it with two complementary tests that hold against real content:
- test_non_opted_packs_default_false:
pins that packs that DIDN'T flip the flag (en_minimal_v1,
he_core_cognition_v1, grc_logos_cognition_v1) still surface
definitional_layer=False through the loader. Defends against
a future change accidentally flipping the flag on a non-opted
pack.
- test_opted_packs_carry_flag:
pins that packs that DID flip the flag (en_core_cognition_v1,
en_core_relations_v1) surface definitional_layer=True through
the loader. Proves the substrate's manifest-field propagation
works against real ratified content, not just fixture packs.
Net: +1 test, same intent (substrate ratifies the manifest field
correctly), now with real-content coverage on both sides of the gate.
All 62 ADR-0084 substrate + prompt-diversity tests pass.
12 KiB
ADR-0084 Pack-Content Build Brief
Audience: A fresh dev agent (cheaper/faster tier). You have NO prior context — read this file and docs/decisions/ADR-0084-definitional-layer.md and act from there. Do not invent design decisions; if something is unclear, surface a question rather than guessing.
Mission: Produce the content required by ADR-0084 (Definitional Layer for Lexicon Packs). You are not implementing the schema parser, the composer integration, or any runtime code. You are only producing JSONL pack files and verifying they cohere.
What already exists (do not duplicate)
language_packs/data/
├── en_core_cognition_v1/ ← HAS glosses.jsonl (78 entries)
├── en_core_meta_v1/ ← HAS glosses.jsonl (72)
├── en_core_attitude_v1/ ← HAS glosses.jsonl (40)
├── en_core_polarity_v1/ ← HAS glosses.jsonl (16)
├── en_core_action_v1/ ← HAS glosses.jsonl (26)
├── en_core_temporal_v1/ ← HAS glosses.jsonl (28)
├── en_core_spatial_v1/ ← HAS glosses.jsonl (24)
├── en_core_causation_v1/ ← HAS glosses.jsonl (15)
├── en_core_quantitative_v1/ ← HAS glosses.jsonl (24)
├── en_core_relations_v1/ ← lexicon only, no glosses
├── en_core_relations_v2/ ← lexicon only, no glosses
├── en_core_relations_v3/ ← lexicon only, no glosses
├── en_collapse_anchors_v1/ ← lexicon only, no glosses
├── he_core_cognition_v1/ ← lexicon only, no glosses
├── grc_logos_cognition_v1/ ← lexicon only, no glosses
├── grc_logos_micro_v1/ ← lexicon only, no glosses
├── he_logos_micro_v1/ ← lexicon only, no glosses
└── en_minimal_v1/ ← lexicon only, no glosses (skip — staging)
Existing glosses.jsonl entries have the flat shape:
{"lemma":"analogy","gloss":"a mapping of relations shared between distinct domains","pos":"NOUN","provenance_ids":["seed:glosses_v1"]}
They DO NOT yet carry definitional_atoms or predicates_invited — those fields are what ADR-0084 introduces. You will extend each existing entry with those two fields, in place, preserving every other field byte-identically.
Phase 1 — Primitives pack (highest priority, blocks everything else)
Create packs/primitives/en_semantic_primitives_v1/. This is a new top-level directory; create it if it does not exist.
Files to create
packs/primitives/en_semantic_primitives_v1/manifest.jsonpacks/primitives/en_semantic_primitives_v1/primitives.jsonlpacks/primitives/en_semantic_primitives_v1/README.md(one paragraph)
Primitives discipline
A primitive is a word whose meaning is taken as terminal at the system level — it does not require a gloss because every other definition can bottom out on it. Pick ~40–60 primitives across these tight categories:
| Category | Examples |
|---|---|
| Existence | exist, be, not_be |
| Identity | same, different, this, that |
| Relation | relation, cause, part, whole, connected_to |
| Mode | way, means, medium |
| Disclosure | visible, hidden, seen, known |
| Quantity (basic) | one, many, none, some |
| State | change, still, start, end |
| Reference | who, what, where, when, why, how |
| Communication | say, mean, ask, answer |
| Logic | if, then, and, or, because |
Avoid overlap with the existing 9 core packs — primitives must be the floor, not duplicates of vocabulary already glossed elsewhere. If you see a word in any existing glosses.jsonl, prefer to leave it out of primitives.
Primitive entry schema
{
"lemma": "exist",
"category": "existence",
"pos": "VERB",
"primitive_version": 1,
"provenance_ids": ["adr-0084:reviewed:2026-05-20"]
}
No gloss. No definitional_atoms. Primitives are terminal.
Manifest
{
"pack_id": "en_semantic_primitives_v1",
"language": "en",
"kind": "primitives",
"definitional_layer": true,
"version": 1,
"issued_at": "2026-05-20T00:00:00Z",
"checksum": "<sha256 of primitives.jsonl as written>",
"primitive_count": <N>,
"never_auto_mutable": true,
"provenance": "adr-0084:reviewed:2026-05-20"
}
Compute checksum as hashlib.sha256(Path("primitives.jsonl").read_bytes()).hexdigest() AFTER writing the file (the manifest hashes the bytes actually on disk).
Acceptance for Phase 1
- File exists at the path above.
- 40 ≤
primitive_count≤ 60. - Every line of
primitives.jsonlis valid JSON with the schema above. - No primitive lemma appears in any existing
glosses.jsonl(run a quickgrep -l "\"lemma\":\"<word>\"" language_packs/data/*/glosses.jsonlfor each — should return empty). - Manifest
checksummatchessha256ofprimitives.jsonlbytes.
Phase 2 — Extend existing 9 core glosses to ADR-0084 schema
For each pack in language_packs/data/ that already has a glosses.jsonl, extend each entry to:
{
"lemma": "analogy",
"gloss": "a mapping of relations shared between distinct domains",
"pos": "NOUN",
"definitional_atoms": ["mapping", "relation", "shared", "distinct", "domain"],
"predicates_invited": ["reveals", "illustrates", "maps"],
"definition_version": 1,
"provenance_ids": ["seed:glosses_v1", "adr-0084:reviewed:2026-05-20"]
}
definitional_atoms rule
Every content word in gloss (skip articles, prepositions, "is", "of", "to", "by", "the", "a", "an", "that", "which") must be in definitional_atoms. Each entry in definitional_atoms must resolve to EXACTLY ONE of:
- Another lemma in the SAME pack's
lexicon.jsonlorglosses.jsonl. - A lemma in another pack's
lexicon.jsonlorglosses.jsonlunderlanguage_packs/data/. - A primitive in
packs/primitives/en_semantic_primitives_v1/primitives.jsonl.
If a word in the gloss does not resolve, do ONE of:
- Rephrase the gloss using a word that does resolve (preferred).
- Add the unresolved word to the primitives pack (only if it is genuinely terminal — surface a note in your final report listing every word you promoted to primitive).
- Add a new gloss to the most appropriate existing core pack (e.g. if "mapping" belongs in
en_core_action_v1, add it there with its own closure).
NEVER invent a new pack just to hold one stray word.
predicates_invited rule
List 2–6 predicates that this lemma legitimately appears with as the subject. Examples:
light→["reveals", "illuminates", "shines", "exposes"]knowledge→["requires", "grounds", "supports", "informs"]parent→["raises", "begets", "guides", "shelters"]
If you are unsure for a given lemma, set predicates_invited: [] (the v1 spec accepts empty during migration). Prefer empty over guessing.
Acceptance for Phase 2
- Every pre-existing
glosses.jsonlentry gainsdefinitional_atoms,predicates_invited,definition_version, and an additionaladr-0084:reviewed:2026-05-20provenance entry. - Run the closure check (Phase 4) and fix any unresolved references before moving on.
- Each pack's
manifest.jsonis updated:- Add
"definitional_layer": true. - Refresh
checksumagainst the newglosses.jsonlbytes. - Append
adr-0084to provenance.
- Add
Phase 3 — New glosses for packs that lack them
Add glosses.jsonl to each of these packs, one entry per lemma in the pack's lexicon.jsonl:
en_core_relations_v1,v2,v3(kinship —parent,child,sibling,family,ancestor,descendant,spouse,offspring, and the v2/v3 additions)en_collapse_anchors_v1(only 3 lemmas — small)
Use the same schema as Phase 2. Same closure rule. Same predicates_invited discipline.
DO NOT touch these in this brief:
he_core_cognition_v1,grc_logos_cognition_v1,grc_logos_micro_v1,he_logos_micro_v1— Greek/Hebrew per-lens glosses are a separate ADR (anchor-lens substrate, deferred per ADR-0084 scope limits).en_minimal_v1— staging pack, not production.
Acceptance for Phase 3
- Each named pack has a
glosses.jsonlwith one entry perlexicon.jsonlentry. - All Phase 2 acceptance items apply.
Phase 4 — Closure verification
Write a stand-alone Python script at scripts/verify_definitional_closure.py that:
- Loads every
glosses.jsonlunderlanguage_packs/data/whose pack manifest has"definitional_layer": true. - Loads
packs/primitives/en_semantic_primitives_v1/primitives.jsonl. - For each gloss entry, for each token in
definitional_atoms, checks it resolves to ONE of (a) same-pack lemma, (b) other-pack lemma, (c) primitive. - Prints a report:
- total entries checked
- total
definitional_atomstokens checked - unresolved tokens grouped by pack
- exit code 0 iff zero unresolved
- Optional
--jsonflag for machine-readable output.
This script is the ratification gate proxy until the real one ships in the schema parser ADR. Run it after every batch of edits.
Acceptance for Phase 4
- Script exists, is runnable as
python -m scripts.verify_definitional_closure. - Exit code 0 on the final pack state.
- A
--jsoninvocation produces a parseable report.
Phase 5 — Run the safety lanes
After every phase, run these and confirm green:
PYTHONPATH=. python3 -m core.cli test --suite smoke -q
PYTHONPATH=. python3 -m core.cli test --suite packs -q
PYTHONPATH=. python3 -m core.cli test --suite cognition -q
PYTHONPATH=. python3 -m core.cli eval cognition
If any of these breaks, STOP and surface the failure. Do not patch around it — the composer was not changed by this ADR, so any cognition/teaching regression means a pack content drift broke an existing invariant. The fix is to the pack content, not to the runtime.
Acceptance for Phase 5
- All four green.
core eval cognitionmetrics byte-identical to pre-brief baseline (capture the baseline FIRST — run the eval before starting Phase 1 and save the output for diff).
Things you must NOT do
- DO NOT modify any file under
chat/,core/,generate/,vault/,field/,algebra/,teaching/,evals/(except for reading them). - DO NOT modify
language_packs/compiler.pyorlanguage_packs/data/<pack>/manifest.jsonfields other thanchecksum,provenance, and the newdefinitional_layerflag. - DO NOT introduce any LLM-generated glosses by routing through an external API. Every gloss is hand-written, drawn from primitives + co-pack vocabulary. If a lemma is genuinely outside your domain knowledge, surface it as a question instead of guessing.
- DO NOT touch Greek / Hebrew packs (per ADR scope limit).
- DO NOT add
predicates_invitedyou cannot defend — empty is the correct default for uncertainty. - DO NOT skip checksum refresh — packs are content-addressed; stale checksums fail ratification.
Reporting
When you are done (or blocked), produce a single markdown report:
# ADR-0084 Pack-Content Build — Final Report
## Phase 1 — primitives pack
- created: <path>
- primitive_count: <N>
- categories covered: <list>
- words promoted to primitive (during Phase 2 closure): <list>
## Phase 2 — existing glosses extended
- packs touched: <list>
- entries extended: <total>
- entries left with predicates_invited=[]: <count>
- closure status: <verified | issues>
## Phase 3 — new glosses created
- packs touched: <list>
- entries added: <total>
## Phase 4 — closure verifier
- script path: <path>
- final exit code: <0 | nonzero>
## Phase 5 — safety lanes
- smoke: <pass | fail>
- packs: <pass | fail>
- cognition: <pass | fail>
- eval cognition metrics diff: <byte-identical | drift details>
## Open questions / handoff items
- <list anything you couldn't answer without guessing>
Surface this report at the end. Do not commit. The human reviews the report before any merge.
Sanity primitives (a starter list — extend, don't blindly copy)
You may use these as a starting point for the primitives pack. Confirm each is NOT already in an existing glosses.jsonl before including:
exist, be, not_be, same, different, this, that, relation, cause, part,
whole, connected_to, way, means, medium, visible, hidden, seen, known,
one, many, none, some, change, still, start, end, who, what, where,
when, why, how, say, mean, ask, answer, if, then, and, or, because,
through, from, into, between, before, after
That's 49. Decide which deserve to be in the primitives pack and which are better placed inside an existing core pack (e.g. through, from, into, between, before, after are likely already in en_core_spatial_v1 or en_core_temporal_v1).
End of brief.