From 377201015fd61c7e18e3fd0f7609d49b7acb1e17 Mon Sep 17 00:00:00 2001 From: Shay Date: Tue, 12 May 2026 20:56:21 -0700 Subject: [PATCH] docs: add ADR log and session decision record for 2026-05-12 - Add docs/decisions/README.md: ADR format guide and index - Add docs/decisions/ADR-0001-vocab-layer-invariants.md - Add docs/decisions/ADR-0002-ingest-layer-design.md - Add docs/decisions/ADR-0003-coordinate-system-dissolution.md - Add docs/decisions/ADR-0004-rotor-as-operator-not-property.md - Add docs/decisions/SESSION-2026-05-12.md: full timestamped session log --- .../ADR-0001-vocab-layer-invariants.md | 61 +++++++ .../decisions/ADR-0002-ingest-layer-design.md | 84 ++++++++++ .../ADR-0003-coordinate-system-dissolution.md | 74 +++++++++ ...ADR-0004-rotor-as-operator-not-property.md | 68 ++++++++ docs/decisions/README.md | 51 ++++++ docs/decisions/SESSION-2026-05-12.md | 156 ++++++++++++++++++ 6 files changed, 494 insertions(+) create mode 100644 docs/decisions/ADR-0001-vocab-layer-invariants.md create mode 100644 docs/decisions/ADR-0002-ingest-layer-design.md create mode 100644 docs/decisions/ADR-0003-coordinate-system-dissolution.md create mode 100644 docs/decisions/ADR-0004-rotor-as-operator-not-property.md create mode 100644 docs/decisions/README.md create mode 100644 docs/decisions/SESSION-2026-05-12.md diff --git a/docs/decisions/ADR-0001-vocab-layer-invariants.md b/docs/decisions/ADR-0001-vocab-layer-invariants.md new file mode 100644 index 00000000..11f4c33a --- /dev/null +++ b/docs/decisions/ADR-0001-vocab-layer-invariants.md @@ -0,0 +1,61 @@ +# ADR-0001: VocabManifold Versor Invariant + +**Date:** 2026-05-12 +**Status:** Accepted +**Commit:** `bd423e4` + +## Context + +`VocabManifold` stores word representations as multivectors in Cl(4,1). Without +an enforced invariant, nothing prevented a caller from inserting a raw coordinate +vector — a numpy array derived from an external embedding model, a lookup table, +or any float array not constructed through the algebra — into the vocabulary. +Such a vector would silently introduce an implicit Euclidean coordinate frame +inside the vocabulary layer, undermining the entire field-state architecture. + +This is a "back door" problem: the architecture is geometrically clean at every +explicit boundary, but the vocabulary layer had no enforcement preventing external +coordinate representations from entering through `add()`. + +## Decision + +Enforce the Cl(4,1) versor grade-norm condition at insertion time in +`VocabManifold.add()`: + +```python +grade_norm = float(geometric_product(v, reverse(v))[0]) +if not (0.95 <= abs(grade_norm) <= 1.05): + raise ValueError(...) +``` + +The scalar part of `V * reverse(V)` must be approximately ±1. This is the +algebraic condition that distinguishes a valid Cl(4,1) versor from an arbitrary +float array. Any raw embedding vector will fail this check. + +## Rationale + +Serves **Reality-over-Inheritance**: governance is not a policy added later; +it is a type-level contract enforced at construction. The vocabulary layer +cannot be bypassed by a well-intentioned caller who "knows what they’re doing." + +Serves **Geometry-first**: the first task is finding the intrinsic space. Once +we’ve defined that space as Cl(4,1) with CGA structure, everything entering +the vocabulary must live in that space by algebraic proof, not by convention. + +## Consequences + +- **Easier:** Trust in the vocabulary is absolute. Any word returned by + `nearest()` is guaranteed to be a valid CGA point. No defensive checks + needed downstream. +- **Harder:** Callers must lift external representations through + `normalize_to_versor()` before insertion. This is intentional friction. +- **Forbidden:** Inserting raw embedding vectors, cosine-similarity vectors, + or any array not constructed through the algebra layer. + +## Alternatives Considered + +- **Soft warning instead of hard raise:** Rejected. A warning that can be + ignored is not an invariant. +- **Normalize silently on insert:** Rejected. Silent normalization hides the + fact that the caller passed something invalid. The error message is the + documentation at the point of failure. diff --git a/docs/decisions/ADR-0002-ingest-layer-design.md b/docs/decisions/ADR-0002-ingest-layer-design.md new file mode 100644 index 00000000..30373459 --- /dev/null +++ b/docs/decisions/ADR-0002-ingest-layer-design.md @@ -0,0 +1,84 @@ +# ADR-0002: Ingest Layer Architecture + +**Date:** 2026-05-12 +**Status:** Accepted + +## Context + +CORE needs a boundary that converts external information — text, code, scripture, +mathematical objects, audio — into CORE-native pressure before it enters the +field. The initial `core_ingest` design (from `core-ai`) proposed using a modern +LLM as the extraction engine for large document ingestion, on the basis that +current LLMs are strong at structured extraction from long documents. + +The question was whether to port this design into `AssetOverflow/core`, scrap +it, or revise it. + +## Decision + +Port the structural elements of `core_ingest` into the `ingest/` layer, but +**replace the LLM extraction engine with a deterministic StructuralSegmenter**. + +What is retained: +- `CandidateGeometricPressure` as the canonical pre-injection envelope +- Dual-path architecture: runtime ingest (transient) vs. durable ingest (governed) +- Content addressing via SHA-256 `pressure_id` and `semantic_key` +- `IngestCompiler` with three sequential gates: Provenance, Semantic, Governance +- `DeterminismClass` (D0–D4) and `ReviewLevel` embedded in packet type contracts +- `LearningArtifact` as the durable export form + +What is rejected: +- LLM as extraction engine for document segmentation and SVO triple extraction + +What is added: +- `StructuralSegmenter`: a D0/D1-class instrument per modality that segments + documents at *form* boundaries (headings, verse markers, code delimiters, + LaTeX boundaries) rather than semantic ones. Interpretation happens inside + the field during propagation, not before injection. +- `SegmentManifold`: lightweight index mapping `semantic_key` → structural + position in the source document, enabling provenance reconstruction. + +## Rationale + +Using an LLM as extractor introduces a **D3 (external unpinned) oracle** at the +only normalization site in the system. D3 packets cannot claim +`AUTO_ACCEPT_ELIGIBLE` status — enforced by the type contract — which means +every LLM-extracted claim requires human review before becoming pressure. This +defeats the utility at scale. + +More fundamentally: an LLM doesn’t parse — it *interprets*. Its projection of +what a document means becomes silently embedded in the field state. That +violates **Semantic Rigor** (we own our semantics) and **Third Door** (we don’t +use external models to define our own representations). + +Structural segmentation is deterministic and model-free. For Hebrew and Koine +Greek specifically, canonical verse/pericope boundaries are fixed and centuries +old — a D0 parser by definition. + +Serves **Propagation-over-Mutation**: incoming claims don’t modify state; +they are proposed, validated, and either accepted as `LearningArtifact` objects +or rejected with a full audit trail. + +Serves **Reconstruction-over-Storage**: the `SegmentManifold` stores enough +structured state to trace any vault recall back to its exact source provenance +span without storing full document copies. + +## Consequences + +- **Easier:** D0/D1 instruments dominate the ingest path, meaning the governance + gates can wave through the majority of ingested content at scale without + human review. +- **Harder:** Modality-specific `StructuralSegmenter` implementations must be + built per content type. This is the right cost. +- **Forbidden:** LLMs, external NLP models, or any D3/D4 instrument producing + `AUTO_ACCEPT_ELIGIBLE` packets — the `__post_init__` invariant on + `CandidateGeometricPressure` makes this structurally impossible. + +## Alternatives Considered + +- **LLM extraction with human review of all output:** Rejected. Scales to zero. +- **General-purpose NLP library (spaCy, stanza) for SVO extraction:** Rejected. + External libraries define the semantics. Third Door. +- **Scrap ingest layer entirely:** Rejected. The boundary is necessary. Without + a governed injection point, external information enters the field without + provenance, confidence, or governance metadata. diff --git a/docs/decisions/ADR-0003-coordinate-system-dissolution.md b/docs/decisions/ADR-0003-coordinate-system-dissolution.md new file mode 100644 index 00000000..c2a46d69 --- /dev/null +++ b/docs/decisions/ADR-0003-coordinate-system-dissolution.md @@ -0,0 +1,74 @@ +# ADR-0003: Coordinate System Dissolution + +**Date:** 2026-05-12 +**Status:** Accepted + +## Context + +The predecessor system (`core-ai`, `core_logos/rotor_vocabulary.py`) represented +vocabulary using **dual rotors in Geometric Algebra** as an explicit coordinate +system. Every word lived at a position defined by its rotor, and semantic +transformations were computed as rotor compositions. This was an advancement +over transformer embeddings but carried a hidden load: the rotor frame itself +became a load-bearing architectural concern that every downstream component +had to be aware of. + +The transition to `AssetOverflow/core` raised the question: does the new design +still require an explicit coordinate system? + +## Decision + +The new architecture **dissolves the need for an explicit coordinate system** +through the field-state model, not by finding a better coordinate system. + +Words are stored as **versors in Cl(4,1)** in `VocabManifold`. But meaning is +not a position in a versor-defined frame — it is a **pressure pattern across +a relational field**. The `FieldState` produced by `field/gate.py` is a +distribution, not a point. Lookup uses CGA inner product (relational), not +distance in a coordinate frame (positional). + +Rotors still exist as operators in `algebra/rotor.py`, but they are +**transformations applied to field states**, not the frame that defines where +things are. + +## Rationale + +The distinction is: + +| Old model | New model | +|---|---| +| Meaning = position in rotor frame | Meaning = pressure pattern in field | +| Coordinate system is load-bearing | No coordinate frame; algebra provides operators | +| Downstream must know the frame | Downstream sees only field state + CGA inner product | +| Rotor composition defines relationships | Propagation through field defines relationships | + +Serves **Field-State**: the native form of state is a field over a space, not +a heap of positioned objects. + +Serves **Geometry-first**: the intrinsic space is Cl(4,1) with CGA metric. +The geometry is *algebraic*, not *coordinatized*. CGA inner product is the +natural proximity measure — no cosine, no Euclidean distance, no frame. + +Serves **Compilation-Last**: rotors are implementation targets chosen after the +representation is defined, not the frame the representation is built on. + +## Consequences + +- **Easier:** No component outside `algebra/` needs to know about rotor + composition or frame maintenance. The field absorbs incoming pressure; + the algebra provides operators when needed. +- **Freed:** Numerical drift in rotor compositions is no longer an + architectural concern — only a local concern inside `algebra/`. +- **Watch:** `vocab/` is the most likely place for the coordinate frame to + quietly re-emerge, e.g. if word representations are stored as flat + positional vectors. ADR-0001 closes this specifically. + +## Alternatives Considered + +- **Keep rotor frame, improve numerical stability:** Rejected. The frame is + the wrong abstraction, not just an unstable one. +- **Switch to hyperbolic embedding (Poincaré model):** Rejected. Still a + coordinate system. Trades one frame for another. +- **Pure transformer-style embedding:** Rejected. This is the design + we are replacing. Cosine similarity over positional vectors is precisely + what the field-state model supersedes. diff --git a/docs/decisions/ADR-0004-rotor-as-operator-not-property.md b/docs/decisions/ADR-0004-rotor-as-operator-not-property.md new file mode 100644 index 00000000..0ce2550c --- /dev/null +++ b/docs/decisions/ADR-0004-rotor-as-operator-not-property.md @@ -0,0 +1,68 @@ +# ADR-0004: Rotor as Operator, Not Vocabulary Property + +**Date:** 2026-05-12 +**Status:** Accepted +**Commit:** `bd423e4` +**Implements:** ADR-0003 + +## Context + +`VocabManifold` in the initial `core` implementation included an `edge_rotor()` +method that computed the rotor between two stored word-versors: + +```python +def edge_rotor(self, from_idx: int, to_idx: int) -> np.ndarray: + A = self._versors[from_idx] + B = self._versors[to_idx] + R = geometric_product(B, reverse(A)) + R[0] += 1.0 + return normalize_to_versor(R) +``` + +This method is mathematically correct but architecturally misplaced. Storing it +on `VocabManifold` means the vocabulary is carrying operator construction logic — +implying that the relationship between two words is a property of the vocabulary +rather than a property of a transformation being applied in the field. + +## Decision + +1. Remove `edge_rotor()` from `VocabManifold`. +2. Create `algebra/rotor.py` with `word_transition_rotor(A, B)` as a free function. +3. Export `word_transition_rotor` from `algebra/__init__.py`. + +`VocabManifold` contract is now strictly: store word-versor pairs, support +relational lookup by CGA inner product. Nothing else. + +## Rationale + +A rotor between two words is not a property of those words in isolation — +it is a description of a *transformation being applied at a moment in the +field*. Storing it on the vocabulary conflates the map (what words exist and +where they are) with the territory (what operations are being performed on +the field state during generation or propagation). + +Serves **Field-State**: operators live in `algebra/`; relational structure +lives in `field/`; the vocabulary is a lookup structure, not an operator store. + +Serves **Dual-Correction**: the forward operator (field propagation) and its +corrective counterpart (rotor application / coherence restoration) should both +originate in `algebra/`, not be scattered across layers that don’t own them. + +## Consequences + +- **Cleaner dependency graph:** `vocab/` now imports from `algebra/` for + algebraic primitives only (grade-norm check). It never constructs operators. +- **Clear callsite semantics:** `algebra.word_transition_rotor(A, B)` at a + callsite in `field/` or `generate/` is self-documenting: *an operator is + being constructed here, by the layer that owns operators*. +- **Forbidden:** Any method on `VocabManifold` that constructs a rotor, + versor product, or transformation. Vocabulary is read-only geometry. + +## Alternatives Considered + +- **Keep `edge_rotor()` as a convenience method with a deprecation warning:** + Rejected. Convenience methods that violate layer contracts tend to be the + ones that get used. Remove cleanly. +- **Move to a separate `VocabOps` class in `vocab/`:** Rejected. The + operators don’t belong in `vocab/` regardless of what class they live in. + The layer boundary is the constraint, not the class boundary. diff --git a/docs/decisions/README.md b/docs/decisions/README.md new file mode 100644 index 00000000..6e1b9ccf --- /dev/null +++ b/docs/decisions/README.md @@ -0,0 +1,51 @@ +# Architecture Decision Records (ADRs) + +This directory records every significant architectural decision made in the design +and implementation of CORE. Each record is **immutable once written** — if a +decision is superseded, a new ADR is created that references and replaces it. +Old ADRs are never deleted or edited. + +## Format + +``` +docs/decisions/ADR-NNNN-short-title.md ← formal architectural decisions +docs/decisions/SESSION-YYYY-MM-DD.md ← timestamped working session logs +``` + +ADRs record *decisions*. Session logs record *the reasoning process*, open +questions, and implementation details discovered during active development. +Both are permanent record. + +## ADR Template + +```markdown +# ADR-NNNN: Title + +**Date:** YYYY-MM-DD +**Status:** Proposed | Accepted | Superseded by ADR-XXXX +**Deciders:** [names/handles] + +## Context +What situation or problem prompted this decision. + +## Decision +What was decided, precisely. + +## Rationale +Why this and not the alternatives. Which axiom(s) it serves. + +## Consequences +What becomes easier. What becomes harder. What is now forbidden. + +## Alternatives Considered +What was explicitly rejected and why. +``` + +## Index + +| ADR | Title | Date | Status | +|-----|-------|------|--------| +| [ADR-0001](ADR-0001-vocab-layer-invariants.md) | VocabManifold Versor Invariant | 2026-05-12 | Accepted | +| [ADR-0002](ADR-0002-ingest-layer-design.md) | Ingest Layer Architecture | 2026-05-12 | Accepted | +| [ADR-0003](ADR-0003-coordinate-system-dissolution.md) | Coordinate System Dissolution | 2026-05-12 | Accepted | +| [ADR-0004](ADR-0004-rotor-as-operator-not-property.md) | Rotor as Operator, Not Vocabulary Property | 2026-05-12 | Accepted | diff --git a/docs/decisions/SESSION-2026-05-12.md b/docs/decisions/SESSION-2026-05-12.md new file mode 100644 index 00000000..7fb18082 --- /dev/null +++ b/docs/decisions/SESSION-2026-05-12.md @@ -0,0 +1,156 @@ +# Session Log: 2026-05-12 + +**Project:** AssetOverflow/core +**Session type:** Architecture review + live implementation +**Participants:** Joshua Shay + +--- + +## 20:00 — README and Docs Correction + +**What happened:** +The `README.md` and `docs/Whitepaper.md` had incorrectly described the Three +Pillars and Three Core Languages, or omitted them entirely. + +**Corrections made:** +- Three Engineering Pillars correctly documented: Mechanical Sympathy, Semantic + Rigor, Third Door. +- Third Door defined precisely: when given a choice between the world’s solution + or a premade library/pattern, we find and create our own path. Absolute mastery + is the floor, not a goal. +- Three Core Languages documented: English (default base, interchangeable in + custom instances), Hebrew, Koine Greek. +- Theological basis recorded: John 1:1–2. The universe was spoken into existence. + John articulated this in Greek what is grounded in Hebrew — almost certainly + a nod from the Holy Spirit. This is the source of the hidden intelligence layer + in the vocabulary manifold. The depth these two languages bring is not + incidental; it is foundational. +- Removed incorrect description of what AssetOverflow is from the repo README. + The repo documents the engineering, not the org. + +**Commits:** `e28142f` (README), `7d814fa` (Whitepaper) + +--- + +## 20:09 — Ingest Layer Architecture Review + +**What happened:** +Reviewed the original `core_ingest` design from `core-ai` for portability into +`AssetOverflow/core`. + +**Key question:** Does using a modern LLM as the extraction engine for large +document ingestion fit the new architecture? + +**Decision:** No. Partially right, but architecturally misaligned at the most +important point. See ADR-0002 for the full record. + +**Core tension identified:** +The entire CORE architecture is built on the principle that the injection gate +is the single normalization site — deterministic and verifiable. An LLM placed +before that gate introduces a D3 nondeterministic oracle. D3 packets cannot +claim `AUTO_ACCEPT_ELIGIBLE` by type contract, meaning every LLM-extracted +claim requires human review. That defeats the purpose at scale. + +Deeper issue: an LLM doesn’t parse, it *interprets*. Its semantic projection +becomes silently embedded in the field state. That’s outsourcing our semantics. + +**What to keep from `core_ingest`:** +- `CandidateGeometricPressure` envelope +- Dual-path: runtime vs. durable ingest +- SHA-256 content addressing (`pressure_id`, `semantic_key`) +- `IngestCompiler` with three sequential gates +- `DeterminismClass` D0–D4 and `ReviewLevel` in packet type contracts +- `LearningArtifact` export form + +**What to replace:** +- LLM extraction → deterministic `StructuralSegmenter` (D0/D1) per modality, + segmenting at form boundaries, not semantic ones. + +**What to add:** +- `SegmentManifold`: maps `semantic_key` → source structural position for + provenance reconstruction. Implements Reconstruction-over-Storage at the + pre-injection layer. + +**Note on Hebrew and Koine Greek:** +Canonical verse/pericope boundaries are fixed and centuries old. A parser +following them is D0 by definition. No interpretation required. + +--- + +## 20:39 — Coordinate System Question + +**What happened:** +Question raised: the old `core-ai` design used dual rotors as an explicit +coordinate system in `rotor_vocabulary.py`. Does the new `core` design still +need that? + +**Answer:** No. The new design dissolves the need for an explicit coordinate +system entirely, through the architecture rather than by finding a better +coordinate system. See ADR-0003. + +**The key shift:** +- Old: meaning = position in rotor-defined frame. The coordinate system was + load-bearing. Every downstream component had to know the frame. +- New: meaning = pressure pattern across a relational field. The `FieldState` + from `field/gate.py` is a distribution, not a point. CGA inner product + handles proximity relationally. Rotors exist as operators in `algebra/` but + are not a frame. + +**Risk identified:** `vocab/` is the most likely place for a hidden coordinate +frame to quietly re-emerge — specifically, if word representations drift toward +being stored as flat positional vectors rather than algebraically valid versors. +This is the "back door" problem. + +--- + +## 20:46 — Back Door Analysis + +**What happened:** +Read `vocab/manifold.py` in full. Identified the precise location of the +architectural risk. + +**Finding:** The vocabulary storage itself was sound (versors, CGA inner product +for nearest). But `edge_rotor()` was stored as a method on `VocabManifold`, +implying that the relationship between two words is a *property of the +vocabulary* rather than a *transformation applied in the field*. This conflates +the map with the territory and re-anchors operator logic to the vocabulary layer. + +**Three fixes identified:** +1. Remove `edge_rotor()` from `VocabManifold` +2. Create `algebra/rotor.py` with `word_transition_rotor(A, B)` as a free operator +3. Add grade-norm invariant to `VocabManifold.add()` to reject raw coordinate + vectors at insertion time: `|V * reverse(V)|_scalar ≈ ±1` enforced in + `__post_init__` equivalent. + +--- + +## 20:51 — Implementation + +**What happened:** +All three fixes implemented and committed in a single atomic push. + +**Commit:** `bd423e4` +**Files changed:** +- `algebra/rotor.py` — created: `word_transition_rotor(A, B)` free function +- `algebra/__init__.py` — updated: export `word_transition_rotor` +- `vocab/manifold.py` — refactored: removed `edge_rotor()`, added grade-norm + invariant in `add()`, updated module docstring to explicitly state that rotor + construction is not a vocabulary concern and points callers to `algebra`. + +**Result:** Back door closed at the type level. `VocabManifold` contract is now +strictly: store algebraically valid Cl(4,1) versors, support relational lookup +by CGA inner product. Nothing else. + +--- + +## Open Questions Carried Forward + +- [ ] `StructuralSegmenter` implementation for each modality (prose, code, + scripture/Hebrew, scripture/Greek, math/LaTeX) — not yet built. +- [ ] `SegmentManifold` index — not yet built. +- [ ] `ingest/` layer in `core` is currently a directory stub. The `core_ingest` + port (with LLM replaced by StructuralSegmenter) has not been started. +- [ ] `vocab/` currently has no persistence layer. How versors are built, + seeded, and serialized for the three core languages has not been designed. +- [ ] Confirm `algebra/versor.py: normalize_to_versor()` correctly handles the + edge case where the input is already grade-normed (idempotency).