core/docs/research/reader-hamiltonian-compiler-spike-2026-07-18.md

Reader → Hamiltonian Compiler — Design Spike

Status: COMPLETE — design ready for review; all bindings verified against the live tree, algebra claims verified in-tree. Implementation (P1–P5) starts after the §8 rulings. Date: 2026-07-18 Base: forgejo/main @ 6ff73aa7 (intelligence-loop arc merged; ADR-0246/0247/0248 Accepted) Branch: feat/reader-hamiltonian-compiler Provenance: Next-arc candidate #1 from docs/handoff/ADR-0246-Acceptance-Evidence.md; the generalized-lift instrument recorded GSM8K/NL non-ingestibility as the composition frontier — this spike designs the mechanism that crosses it.

---

1. Problem statement

The corridor can solve what it can ingest, and it can ingest almost nothing. Today's compilable domains are (a) propositional entailment over ≤5 atoms and (b) quadratic wells around a known target versor. Real problems — GSM8K arithmetic, multi-step deduction — have no path onto the substrate. The measurement harness (Phase 4 instrument) is built and waiting; the compiler is the missing feeder. This is the recorded composition wall from the GSM8K learning history finally getting its mechanism.

Done-when (spike): a ratifiable design that (1) compiles real reader output into corridor Hamiltonians deterministically, (2) provably encodes relations, not answers (anti-hollow invariant, §4.1), (3) reuses the merged Ring-1/2/3 machinery for composition, and (4) names its governing ADR obligations line-by-line. Implementation phases follow only after the design is reviewed.

2. Design thesis: composition = certified turn sequences

The wall is COMPOSITION (ADR-0191/0192/0193 lesson: single-layer widening is metric-inert). The freshly merged arc supplies exactly the composition machinery: per-turn RelaxationCertificates, egress routing, the Ring-2 residual protocol's replay-verified path ledger, Ring-3 handoff, and the S3 tether. Therefore:

The compiler does not compile a problem into one big Hamiltonian. It compiles a problem into a turn program — an ordered DAG of small relation-Hamiltonians, each solvable in one lifecycle turn, chained through the Ring-2 path ledger.

• Each turn's H is ≤ the substrate's native 32-dim space (the ≤5-atom limit is 2⁵ = 32 — the truth-table basis saturates the Cl(4,1) blade basis; this is a feature to design around, not an accident).
• Composition depth lives in the certified chain, not in matrix size. Multi-step problems become multi-turn programs whose end-to-end integrity is the path ledger's job — the same ledger that already discriminates port:identity:d_stab>epsilon_turn.
• This unifies the two flagship domains under one design: arithmetic chains (each step = one relation well) and >5-atom deduction (ROBDD-partitioned ≤5-atom slices per turn).

3. Substrate-native quantity encoding (the standing hand)

Numbers must live on the field before arithmetic relations can be field constraints. The CGA-native choice — no bespoke machinery — is the conformal embedding of the real line:

• Quantity: q ↦ P(q) = e_o + q·e₁ + ½q²·e_∞ (null point on the line's horosphere).
• Addition by a known constant: translator versor T_a = 1 − ½ a e₁e_∞; T_a P(b) T̃_a = P(a + b) — exact, versor-native.
• Scaling by a known constant: dilator versor D_s; D_s P(b) D̃_s = P(s·b) — exact.
• Subtraction / division: inverse versors of the above.
• Relation as constraint: "x = a·y + b" compiles to a quadratic well whose minimum is the versor-transported target — i.e. the existing quadratic-well machinery centered on (T_b D_a) ψ_y (T_b D_a)~, never on a precomputed number.
• Weight honesty: versor transport of null points is exact up to conformal weight (dilators rescale the point's overall factor), and the corridor requires unit-norm states (compile_quadratic_well refuses target_not_unit at 1e-9). The compiler therefore normalizes transported targets, and quantity decode is projective — q is recovered from coefficient ratios (e₁-coefficient over e_o-coefficient), which are scale-invariant, so normalization loses nothing.

Scope honesty (Tier 1): products of two unknowns are not versor operations. Tier-1 scope is affine DAGs — each step multiplies/divides by problem-text constants and adds/subtracts known or previously-certified quantities. This covers the dominant GSM8K step shape; the non-affine remainder is recorded, not silently capped (§6 risks).

Verified in-tree (2026-07-18): all four algebra claims checked numerically against algebra/cl41.py's own multiplication table (signature (+,+,+,+,−), basis e1..e5; n_∞ = e4+e5, n_o = ½(e5−e4)), 200 random cases, q ∈ [−50, 50]: nullity |P(q)²| < 3e-10; translator exactness |T_a P(b) T̃_a − P(a+b)| < 2e-10 (f64 rounding only — the identity is algebraically exact, translators preserve weight); dilator D = exp(+α/2·e4e5) scales s = e^{−α} (sign convention pinned here) with conformal weight rescale confirmed (w ≈ 2.01 ≠ 1 at α=0.7 — normalization required); projective decode q = e1c/(e5c − e4c) scale-invariant to < 1e-9.

4. Design invariants (pins before code)

4.1 Relation-not-answer (anti-hollow)

The compiler must never evaluate the arithmetic or logic it encodes. If the compiler computes a+b to center a well, the corridor confirms rather than solves — a hollow instrument. Enforcement: (i) purity pin — no arithmetic on problem quantities in the compiler module beyond versor construction; (ii) property test — compiled artifacts for a relation are invariant to which variable is designated unknown; (iii) ablation test — corrupting the relaxation changes the answer while compiler output is bit-identical.

4.2 Determinism + content addressing

Compiled Hamiltonians are frozen, content-addressed per ADR-0245 §2.3/§2.4 (full SHA-256, canonical JSON, explicit <f8 LE bytes; _cached_eigh keying unchanged). Same parse ⇒ same hamiltonian_id, byte-identical.

4.3 Fail-closed compilation

Unsupported relation shapes, unparseable steps, or out-of-scope structures refuse with typed errors in the existing HamiltonianCompileError family (cognitive_lifecycle.py already carries the taxonomy: bad_shape, not_symmetric, target_not_unit, atom_count_out_of_range, …) — extend that family upstream rather than inventing a parallel one. Never a guessed or partial Hamiltonian.

4.4 Reader boundary (fix upstream)

The compiler consumes the existing readers' typed output only. If a reader's output shape is insufficient, extend the reader's typed surface upstream — the compiler never parses raw text.

4.5 Governance

Evals-quarantined (A-04): the compiler and turn-program executor live off-serving; serving consumes nothing from this arc. Sealed practice vs serving split per ADR-0175; wrong=0 guard held on the deductive flagship; honest-NULL protocol on the instrument; proposal-only learning (I-03) untouched.

4.6 Byte-identity & cross-hardware reproducibility (three tiers, honestly bounded)

The determinism claim splits into three tiers; only the first two are unconditionally achievable, and the design says so rather than promising universal bit-identity.

• Tier 1 — content address: SOLVED, unconditional. hamiltonian_id = SHA-256 over canonical JSON + explicit <f8 LE matrix bytes (ADR-0244 §2.7). Endianness — the one real architecture variable in hashing — is already coerced. Same matrix ⇒ same id on any machine.
• Tier 2 — matrix construction: ACHIEVABLE, enforced in P1. algebra.cl41.geometric_product (:108-125) is a fixed-order scatter-add (result[idx] += sign·aᵢ·bⱼ, canonical i,j order), no BLAS / dot / reduction ⇒ IEEE-754 correctly-rounded and hardware-deterministic. The translator/dilator identities have integer/half-integer coefficients, so well centers are bit-clean (verified §3). Trap: geometric_product and reverse silently fall back to float32 unless handed f64 arrays (:110-112, :134-135). P1 constructs every versor and sandwich in explicit f64, and pins the result with golden-bytes tests — the frozen SHA-256 of a canonical compiled-Hamiltonian set as literal constants, so any numpy / toolchain / hardware drift in the last bit fails the test loudly (same discipline as the biography <f8 hash pin and the _cached_eigh memoization tests).
• Tier 3 — eigensolve → answer: GENUINELY hardware-bounded; do not overclaim. The affine well H = c(Id − ψψᵀ) is dense, so relax_to_ground takes the LAPACK branch (cognitive_lifecycle.py:590-593). Its excited eigenspace is 31-fold degenerate (all at energy c), and LAPACK fills that basis arbitrarily per build ⇒ evecs/propagator bytes are not cross-hardware identical. Mitigation, in order:
  1. Analytic ground recovery. The compiler constructs the target ψ (versor transport), so the corridor's job is to confirm relaxation reaches span(ψ), not to discover it. The converged steady state projects onto the unique 1-D ground space, so the degenerate-basis arbitrariness cancels in the limit; pin the basis-invariant phase_correlation/2 agreement to ψ, never the propagator bytes.
  2. Where a future non-affine tier yields a genuinely dense non-degenerate H, record cross-hardware eigen-reproducibility as an explicit limitation (no-silent-caps) and pin the decision (_certified already uses tolerances, not bit-equality), not the bytes.

5. In-tree inventory (survey bindings)

5.1 Hamiltonian construction contract (verified)

• ProblemHamiltonian (core/physics/cognitive_lifecycle.py:262-302): frozen 32×32 real-symmetric f64; refuses bad_shape / non_finite_matrix / not_symmetric (≤1e-12, never symmetrized); hamiltonian_id = SHA-256 of canonical JSON over domain + LE-f64 matrix bytes + stringified metadata; is_diagonal auto-derived → diagonal compilers get the no-LAPACK fast path for free (relax_to_ground :581-588).
• compile_quadratic_well (:305-319): H = curvature·(Id − ψ₀ψ₀ᵀ); refuses target_not_unit (1e-9) — hence §3's normalize-then-projective-decode.
• compile_propositional (:382-401) + PropositionalProblem (:326-361, _MAX_ATOMS = 5 at :84): diagonal falsification-count H over the 2⁵ = 32 assignment↔blade bijection derived through real geometric products (_build_subset_component_map :183-203). propositional_entails (:714-750) reads exact integer ground energies — no eigensolve.
• _cached_eigh (:512-530): LRU keyed (hamiltonian_id, matrix bytes); ADR-0245 §3 perf gate (0 allocations / 0 LAPACK on repeated static H) binds compiler emissions.
• Sensorium compilers emit unit-versor field states, never Hamiltonians; the only H built from sensorium data is a quadratic well on the compiled percept (evals/adr_0243_cognitive_lifecycle/__init__.py:114).

5.2 Governing authorities (cite in the ADR)

• ADR-0243 §2.2 — the definition this compiler implements: problem constraints formulated as potential wells in H_problem; "zero room for intermediate fabrication". §2.4/§4.2 — one-mutation-path; speculative H changes quarantined in evals/ until signed certificate.
• ADR-0244 §2.5/§2.7/§2.8 — f64 relaxation, full-SHA content addressing, frozen H.
• ADR-0245 §2.2–2.4, §3 — serving-boundary cast, _cached_eigh, perf gate.
• ADR-0012 — deterministic ingest governance; an LLM/D3 oracle upstream of the gate is explicitly rejected → the compiler's text side must ride the existing deterministic readers.
• ADR-0247 §1 props 3–6 — purity/fail-closed/replay obligations when compiler outputs flow through ports. ADR-0246/0247/0248 are otherwise silent on problem-domain expansion (no conflict). AGENTS.md:99 — no stochastic generation inside the deterministic path.
• ADR-0175 / 0191 / 0192 / 0193 — eval-entry law: sealed-practice vs serving split; completeness obligation (every source quantity consumed or refuse); class-not-list firewall; composition wall (single-layer widening is metric-inert). "ADR-0190" is citation-only — no such file exists; cite the instrument-before-consumption lesson via ADR-0193.
• Numbering: next free ADR number is 0249 (0240–0248 collision-free; re-check at landing).

5.3 Prior art this arc builds ON (reconciliation required)

• ADR-0217 R2 finite-integer constraint compiler (RATIFIED, BUILT, off-serving) — problem text → entities → quantities → relations → CONSTRAINTS → goal → solver → verifier. The reader→Hamiltonian compiler should be designed as a constraint-IR → Hamiltonian backend behind this ratified front-end, not a new text parser (§4.4 fix-upstream).
• docs/research/field-reasoner-wedge-selection.md — standing two-compiler independence design: §6 symbolic compiler, §7 field compiler MUST NOT import the symbolic one, §5 oracle imports neither, §10 GSM8K re-entry rule, §13 stop conditions; chosen wedge = ratio/proportion; state: 0A–0C green, 0D+W next. This arc must either continue that wedge or explicitly absorb it — not silently fork it (§8 decision).
• docs/research/independent-comprehension-agreement-gate.md §3/§7 — two independent readers must construct the same canonical structure before any field-backed promotion. Resolution honoring both constraints: the Hamiltonian backend consumes the post-agreement canonical structure, so reader independence lives upstream of it and the backend imports neither reader.

5.4 Reader stack bindings (verified)

• Arithmetic IR (Tier-1 input): Reader A — generate/math_candidate_graph.py:589 parse_and_solve → MathProblemGraph (generate/math_problem_graph.py:449: entities / initial_state / operations / unknown, typed Operation kinds, canonical bytes). This is the affine DAG §3 compiles. Reader B (generate/derivation/pool.py:69 resolve_pooled) is sealed-practice only and MUST NOT feed the compiler (standing two-reader disjointness; no bridge without sealed wrong=0).
• Deduction IR (Tier-1 input): generate/meaning_graph/reader.py:330 comprehend (NL prose, refusal-first) → projectors.py:153 to_deductive_logic → (premises, query) formula strings. The corridor side consumes PropositionalProblem (CNF tuples, ≤5 atoms). The only missing piece on this leg is a structural formula→CNF converter — syntax-directed distribution over the restricted grammar with typed budget/atom-count refusals, never truth-table construction (which would evaluate, not translate). Today the bridge exists only ad hoc and in reverse (evals/adr_0243_cognitive_lifecycle/ propositional_falsifier.py, CNF → formula strings).
• In-tree precedent for §3's mechanism: the field wedge's generate/relational_field_reader.py:125 already reads relational problems into CGA translator versors (import-disjoint from its symbolic control arm, INV-27). The quantity kernel generalizes this proven mechanism into the corridor; the wedge's own 0D+W trajectory continues unaffected, and any future merge is an explicit ADR decision, not silent absorption.
• Every reader in the tree is refusal-first with typed refusal vocabularies and frozen-slots canonical outputs — the compiler inherits a uniform, well-typed upstream surface.

6. Risks (honest register)

Risk	Level	Handling
Turn-program chaining exposes drift the single-turn arc never exercised	Medium — this is the point	The path ledger + tether are the instruments; failures here are findings, not embarrassments
Tier-1 affine scope covers less of GSM8K than expected	Medium	Coverage measured on the real holdout slice and RECORDED (no silent caps); non-affine remainder scopes Tier 2
Anti-hollow invariant leaks (compiler smuggles evaluation)	High-harm if missed	§4.1 triple enforcement: purity pin + invariance property + ablation test
Quantity decode needs vocab the manifold lacks	Low	Quantity readback decodes null points geometrically, not via word vocab; word-level articulation of results remains gated by vocab coverage (separate arc)
Instrument shows corridor NULL vs symbolic evaluator baseline	High-likelihood, low-harm	Honest NULL recorded; the instrument exists to make this decidable

7. Phase plan (post-review)

• P1 — Quantity kernel (core/physics/): line embedding P(q), translator/dilator construction, normalize + projective decode; property tests for exactness (§3's verified claims become pinned tests), weight-rescale honesty, refusals on non-finite q.
• P2 — Relation compiler v0: affine relation → versor-transported, normalized quadratic-well ProblemHamiltonian (reusing compile_quadratic_well); typed refusals in the HamiltonianCompileError family for non-affine/out-of-scope relations.
• P3 — Formula→CNF converter (deduction leg): syntax-directed CNF over the restricted grammar → PropositionalProblem; typed refusals atom_count_out_of_range (>5) / clause-budget; closes the prose → comprehend → to_deductive_logic → CNF → corridor path.
• P4 — Turn-program compiler + executor (evals quarantine): MathProblemGraph → ordered turn program (one relation well per step, previously-certified quantities as transported targets); executor chains lifecycle turns through the Ring-2 residual protocol (per-turn certificates, replay-verified path ledger); tether readings per §S3.
• P5 — Instrument integration: arithmetic-chain domain added to the generalized-lift instrument; real GSM8K holdout slice, never the 150 templated cases; baseline = the symbolic evaluator over the same MathProblemGraph; honest-NULL protocol; Tier-1 affine coverage of the slice measured and RECORDED.
• P6 (next arc, not this one) — >5-atom deduction via ROBDD-partitioned turn programs.

Each phase: own PR, smoke-gated, TDD-first. New machinery ⇒ ADR-0249 (Proposed; re-verify number at landing), acceptance evidence assembled as in the intelligence-loop arc; no self-Accept.

8. Rulings (RESOLVED 2026-07-18)

All four APPROVED by Shay:

1. Tier-1 = affine-only. Unknown×unknown deferred; keep the versor story exact.
2. >5-atom deduction = next arc, via ROBDD-partitioned turn programs. NOTE the honest reason: this is a sequencing choice (one capability per arc), not a claim that

   5-atom deduction is impossible or needs probabilistic approximation — composition handles it deterministically and exactly. The 32-blade ceiling only forbids doing it natively in one Hamiltonian; the certified turn chain crosses it losslessly.

3. One ADR (0249) for quantity kernel + relation compiler + CNF converter + turn programs.
4. Field wedge continues independently (0D+W); this arc generalizes the same versor mechanism into the corridor; compiler imports neither wedge arm (INV-27 intact); any merge is a later explicit ADR.