docs(decisions): ADR-0005 language pack contract + session addendum [Batch 1]
This commit is contained in:
Shay
parent
377201015f
commit
f68a2dd66c
2 changed files with 244 additions and 0 deletions
167
docs/decisions/ADR-0005-language-pack-contract.md
Normal file
167
docs/decisions/ADR-0005-language-pack-contract.md
Normal file
|
|
@ -0,0 +1,167 @@
|
|||
# ADR-0005: Language Pack Contract
|
||||
|
||||
**Date:** 2026-05-12
|
||||
**Status:** Accepted
|
||||
|
||||
## Context
|
||||
|
||||
CORE operates on three foundational languages: English (default articulation
|
||||
base), Hebrew (depth pack 1), and Koine Greek (depth pack 2). These are not
|
||||
equivalent choices from a menu — they are the three specifically chosen
|
||||
because of the range of depth they bring and because of the theological and
|
||||
linguistic insight encoded in John 1:1–2, where the universe itself was spoken
|
||||
into existence and John articulated that reality in both Hebrew and Greek.
|
||||
This is why and how CORE finds its truth and power in design and
|
||||
communication — being primary, being a core.
|
||||
|
||||
A loose plugin API is insufficient for these packs. The system needs a strict
|
||||
contract that specifies what a language pack must provide, what invariants it
|
||||
must satisfy, and what gates it must pass before activation.
|
||||
|
||||
## Decision
|
||||
|
||||
Adopt a **Language Pack Contract** as the canonical template for all CORE
|
||||
language packs.
|
||||
|
||||
Each pack is a deterministic, versioned manifold bundle that defines:
|
||||
|
||||
- identity and normalization policy
|
||||
- lexical inventory (lemma-first, not token-first)
|
||||
- morphology
|
||||
- syntax and frame templates
|
||||
- semantic lift rules into CORE-native pressure
|
||||
- readback rules from field state to surface language
|
||||
- validation probes and alignment gates
|
||||
|
||||
The three canonical packs are:
|
||||
|
||||
| ID | Name | Role |
|
||||
|-----|--------------------|-----------------------------|
|
||||
| `en`| English | Default articulation base |
|
||||
| `he`| Hebrew | Depth language pack 1 |
|
||||
| `el`| Koine Greek | Depth language pack 2 |
|
||||
|
||||
Each pack must implement the same top-level contract and must target shared
|
||||
field primitives. No pack may define private or opaque semantics that bypass
|
||||
the shared field.
|
||||
|
||||
## Required Structure
|
||||
|
||||
```
|
||||
packs/
|
||||
common/
|
||||
contracts/
|
||||
language-pack.md
|
||||
schema/
|
||||
lemma.schema.json
|
||||
morphology.schema.json
|
||||
frame.schema.json
|
||||
sense.schema.json
|
||||
probe.schema.json
|
||||
anchors/
|
||||
trilingual-anchor-template.json
|
||||
en/
|
||||
pack.toml
|
||||
orthography.yaml
|
||||
lemmas.jsonl
|
||||
morphology.jsonl
|
||||
frames.jsonl
|
||||
senses.jsonl
|
||||
lift_rules.py
|
||||
readback_rules.py
|
||||
validators.py
|
||||
probes/
|
||||
corpora/
|
||||
manifest.yaml
|
||||
he/ (same structure)
|
||||
el/ (same structure)
|
||||
```
|
||||
|
||||
## Validation Gates
|
||||
|
||||
A pack is not considered active because files exist. A pack becomes active
|
||||
only when it passes the following gates in order:
|
||||
|
||||
1. **Schema gate** — all data files validate against their JSON schemas
|
||||
2. **Lexical gate** — lemma inventory is non-empty and internally consistent
|
||||
3. **Morphology gate** — all surface forms trace to a known lemma
|
||||
4. **Lift gate** — lift rules produce valid `CandidateGeometricPressure` for seed inputs
|
||||
5. **Readback gate** — readback rules produce grammatical surface from seed field state
|
||||
6. **Determinism gate** — all D0/D1 paths produce identical output on repeated runs
|
||||
7. **Alignment gate** — pack publishes anchor records verifiable against trilingual template
|
||||
8. **Coverage gate** — probe set covers minimum lexical and morphological surface
|
||||
|
||||
## Design Rules
|
||||
|
||||
### 1. Lemma-first
|
||||
Tokens are surface realizations. Lemmas are the stable lexical unit. The
|
||||
ingest and field-state layers operate on lemmas, not tokens.
|
||||
|
||||
### 2. Deterministic normalization
|
||||
Normalization must be deterministic and script-aware. Unicode normal form NFC
|
||||
is the minimum. Script-specific rules (right-to-left, vowel pointing, accent
|
||||
diacritics) are defined per pack in `orthography.yaml`.
|
||||
|
||||
### 3. Native morphology
|
||||
Morphology is part of meaning-bearing structure, not optional metadata.
|
||||
Hebrew stems (Qal, Niphal, Piel…) and Koine Greek aspects (aorist, perfect…)
|
||||
are semantically load-bearing. Morphology is a first-class pack surface.
|
||||
|
||||
### 4. Shared field target
|
||||
Semantic lift must target shared CORE field primitives. No pack-private
|
||||
semantic space is permitted. What cannot be expressed in shared primitives
|
||||
must be proposed as a new shared primitive, not hidden inside a pack.
|
||||
|
||||
### 5. Readback is owned locally
|
||||
Each pack owns its articulation rules and ambiguity management. Readback is
|
||||
not a global postprocessor — it is part of the pack contract.
|
||||
|
||||
### 6. Alignment is explicit
|
||||
Packs must publish anchor records that are verifiable against the
|
||||
trilingual anchor template. Cross-language coherence is a probe surface,
|
||||
not an emergent hope.
|
||||
|
||||
## Runtime Boundary Honesty
|
||||
|
||||
Files that define a semantic boundary not yet fully specified must raise
|
||||
`NotImplementedError` at exactly that boundary. This is not fake code — it
|
||||
is an honest contract: the structure is real, the interface is real, the
|
||||
unimplemented edge is visible and loud. Nothing is silently skipped.
|
||||
|
||||
## Rationale
|
||||
|
||||
This serves the three engineering pillars:
|
||||
|
||||
- **Mechanical Sympathy** — a small, explicit, cheap-to-validate pack surface
|
||||
that integrates directly with the field-state model.
|
||||
- **Semantic Rigor** — all packs target shared field primitives instead of
|
||||
inventing language-private meanings. Exactness is non-negotiable.
|
||||
- **Third Door** — standard token-plugin patterns were considered and
|
||||
rejected. This contract is designed for CORE's geometry specifically.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Easier:**
|
||||
- Every future language pack follows the same geometry and QA path.
|
||||
- English, Hebrew, and Koine Greek can be aligned by shared anchors and
|
||||
deterministic probes.
|
||||
- Gate failures are explicit and traceable.
|
||||
|
||||
**Harder:**
|
||||
- Pack creation requires real morphology, lift, and readback logic.
|
||||
- Shortcuts that bypass field primitives are structurally forbidden.
|
||||
|
||||
**Forbidden:**
|
||||
- Opaque embedding tables.
|
||||
- Black-box semantics.
|
||||
- Pack-local meaning systems that bypass the shared field.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
- **Loose plugin API:** Rejected. Too weak to enforce invariants.
|
||||
- **Dictionary-only packs:** Rejected. Lexicons alone do not articulate or lift.
|
||||
- **Tokenizer-first packs:** Rejected. Tokens are surface artifacts, not the
|
||||
stable unit of design.
|
||||
- **LLM-as-extraction-engine:** Rejected. Introduces a D3 nondeterministic
|
||||
oracle into the normalization site. See ADR-0002 for the ingest-layer ruling
|
||||
on determinism classification.
|
||||
77
docs/decisions/SESSION-2026-05-12-language-packs-addendum.md
Normal file
77
docs/decisions/SESSION-2026-05-12-language-packs-addendum.md
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
# Session Log Addendum: 2026-05-12 — Language Pack Design
|
||||
|
||||
**Project:** AssetOverflow/core
|
||||
**Session type:** Architecture design addendum
|
||||
**Related ADR:** ADR-0005
|
||||
|
||||
---
|
||||
|
||||
## 21:02 — Language Pack Direction Confirmed
|
||||
|
||||
**What happened:**
|
||||
The decision was made to move from general discussion of language integration
|
||||
toward a formal, meticulous language pack contract for CORE.
|
||||
|
||||
**Why:**
|
||||
The foundational languages are not optional output skins. They are part of
|
||||
the native articulation and semantic design of the system. Therefore the next
|
||||
clean artifact is a strict contract and scaffold, not an informal note.
|
||||
|
||||
**Decision:**
|
||||
Create ADR-0005 and establish the `packs/` structure with shared contracts,
|
||||
schemas, trilingual anchor templates, and initial `en`, `he`, and `el` pack
|
||||
scaffolds.
|
||||
|
||||
---
|
||||
|
||||
## Design Notes Recorded
|
||||
|
||||
### Language choice rationale
|
||||
|
||||
The three languages were chosen because of the range of depth they bring:
|
||||
|
||||
- **English** is the default base (any language could serve this role in a
|
||||
custom CORE instance, but English is the default).
|
||||
- **Hebrew** and **Koine Greek** are the depth languages. John 1:1–2: the
|
||||
universe itself was spoken into existence, and John articulated it in both
|
||||
Hebrew thought and Greek expression — almost certainly a nod from the Holy
|
||||
Spirit. The hidden layer of intelligence in CORE's vocabulary manifold is
|
||||
grounded in this. This is why and how CORE finds its truth and power in its
|
||||
design and communication — being primary, being a core.
|
||||
|
||||
### Pack design decisions
|
||||
|
||||
- Packs are **lemma-first**, not token-first.
|
||||
- Morphology is **native** to the pack contract, especially for Hebrew and
|
||||
Koine Greek where morphology is semantically load-bearing.
|
||||
- **Readback belongs to the pack**, not to a global postprocessor.
|
||||
- Semantic lift must target **shared field primitives**.
|
||||
- Pack activation is **gate-based**, not file-existence-based.
|
||||
- Cross-language alignment is an **explicit probe surface**, not an
|
||||
emergent hope.
|
||||
- Runtime boundary files raise `NotImplementedError` at unimplemented
|
||||
semantic edges — honest structure, not fake behavior.
|
||||
|
||||
### LLM as ingest extraction: rejected
|
||||
|
||||
Using a general-purpose LLM as the extraction engine for large document
|
||||
ingest was considered and explicitly rejected. The reason: it introduces a
|
||||
D3 nondeterministic oracle at the normalization site, and its interpretations
|
||||
are silently embedded in the field state without provenance. Semantic Rigor
|
||||
forbids this. The replacement is a deterministic, structure-aware segmenter
|
||||
(StructuralSegmenter) that carves at form boundaries — document structure,
|
||||
verse markers, code delimiters — not at interpreted content boundaries.
|
||||
For Hebrew and Koine Greek this is even cleaner: canonical verse and pericope
|
||||
boundaries have been fixed for centuries and constitute D0 segmentation.
|
||||
|
||||
---
|
||||
|
||||
## Build Order Established
|
||||
|
||||
| Batch | Contents |
|
||||
|-------|----------|
|
||||
| 1 | ADR-0005, this session addendum |
|
||||
| 2 | `packs/common/` — contract, schemas, anchor template |
|
||||
| 3 | `en`, `he`, `el` pack manifests and orthography files |
|
||||
| 4 | Lexical seed sets — lemmas, morphology, frames, senses, probes |
|
||||
| 5 | Runtime boundary files — lift rules, readback rules, validators |
|
||||
Loading…
Reference in a new issue