From a4f567f0973c498976c33c1fa8818506cebc694e Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 28 Jul 2026 17:02:53 +0000 Subject: [PATCH] =?UTF-8?q?docs(plans,assessment):=20G-24=20=E2=80=94=20th?= =?UTF-8?q?e=20perception=20diagnosis,=20and=20the=20Perception=20Arc=20pl?= =?UTF-8?q?an?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Why capability has not moved despite everything being implemented, answered by measurement rather than opinion, at 0e1be8be: F-A The serving path is 100% symbolic. reader.py (490), proof_chain (3,916), both surface composers: zero geometric references. All served capability comes from a template parser feeding an ROBDD engine. F-B The reader INVERTS its own ratified design. ADR-0252 §2 stage 1 prescribes containment/transfer/accumulation/comparison and bans surface-slot regexes verbatim; the implementation is surface-slot regexes over categorical/member/ordering/propositional. Corpus agrees: 1.28% read rate, possession/transfer/quantity absent, if/then (0.1% of sentences) holding 21% of the inventory. F-C Template architecture cannot produce chunk lifts — each construction is one hand-fitted rule; generalization is zero by construction. This is why lifts have only ever arrived as overfit slivers. F-D Geometry does exactly two cognitive jobs and both are stranded: salience->attention (bypassed by every licensed lane) and relation_compiler's Hamiltonian ground-state solver (off-serving, behind a 1.0% math reader). Real geometric computation exists — it "NEVER evaluates scale*input+offset in Python" — and nothing feeds it. F-E The §5 NO-GO killed alignment-on-points-under-similarity. The composition branch — relations as OPERATORS, meaning as composition, the pattern relation_compiler already embodies — was never tested. Plan: docs/plans/2026-07-28-perception-arc.md. Phase 0 pins the 1.28%/1.0% read-rate floors as the metric the arc is accountable to. Phase 1 is a §5-shaped pre-registered GO/NO-GO on hypothesis H2 (compositional core-primitive reading over operator encodings; NO-GO branch declared: a principled compositional grammar instead — either verdict exits the template treadmill). Phase 2 builds ONE perception layer as combinator core + declared construction lexicon, so coverage growth is table rows, not parsers. Phase 3 reconnects the Hamiltonian corridor to serving through the now-honest licence machinery. Phase 4 deletes what earned it: G-3's template-widening program superseded, curriculum-serve machinery parked. Phase 5 prepares the ADR-0252 amendment and the use_salience ruling for one-word decisions. Explicitly untouched: core/physics, the algebra, the reliability gate, the ledger discipline — correct and load-bearing. The enumerative reader stays serving until its replacement proves out: supersession-in-place, per the ADR's own preserved rule. G-24 registered Tier A. G-3 as previously specified is superseded, recorded in the execution plan's closing addendum rather than silently. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Wcw2pnMBwyvmNyQg4uPEt4 --- docs/assessment/30-gap-register.md | 2 + docs/assessment/50-execution-plan.md | 2 + docs/plans/2026-07-28-perception-arc.md | 127 ++++++++++++++++++++++++ 3 files changed, 131 insertions(+) create mode 100644 docs/plans/2026-07-28-perception-arc.md diff --git a/docs/assessment/30-gap-register.md b/docs/assessment/30-gap-register.md index 2bc77fc4..c0df0f67 100644 --- a/docs/assessment/30-gap-register.md +++ b/docs/assessment/30-gap-register.md @@ -201,6 +201,8 @@ Own `use_salience`, the two underived constants, the self-narrowing budget feedb **The mechanism, and why it is G-7's story again:** one commit updated the lane, its verifier and its test, and missed one downstream artifact. The pin that guards that artifact — `tests/test_claims_md_is_current.py` — **exists and is an orphan in no curated suite**, so nothing surfaced it until this arc ran the full tree once. Registered on the gate as part of the fix. - **G-21 · The math reader decides 1.0% of `holdout_dev/v1`** *(new, 2026-07-28)* — measured while building the §5 corpus: `parse_and_solve` returns a selected graph for **5 of 500** held-out cases, all carrying the same relational skeleton (`compare_multiplicative`); on the public lane, 24/150. This is a sharper measurement of the comprehension frontier than G-3's construction count, on the corpus the project already treats as its held-out standard, and it is why ADR-0252 §5.1's four-structure corpus is not extractable today. Distinct from G-3 (which counts *constructions* the general reader admits); this counts *cases decided* on a standing eval corpus. **Authority:** the widening program (G-3) + a ruling on whether holdout decision-rate becomes a tracked lane metric. - **G-20 · The `refusal_reason` materialisation** — typed refusal evidence exists and is discarded at the public `str` boundary; the plumbing for materialisation already landed. Cross-listed as H-3. **Authority:** small ADR (anticipated by the ADR-0024 chain). +- **G-24 · The perception layer inverts its own ratified design, and no served byte is geometric** *(new, 2026-07-28 — Tier A; the arc-closing diagnosis)* — Measured at `0e1be8be`: the entire serving path (`meaning_graph/reader.py` 490 lines, `proof_chain/` 3,916, both surface composers) contains **zero** geometric references — comprehension is a template parser feeding an ROBDD engine. **ADR-0252 §2 stage 1 prescribes** *"containment / transfer / accumulation / comparison … **Not surface-slot regexes**"* — the implementation is surface-slot regexes over categorical/member/ordering/propositional. The corpus confirms the inversion independently: on `holdout_dev/v1`, **read rate 1.28%** (93.16% `no_template_match`), with possession (18.4% of sentences), transfer verbs (14.6%) and quantity comparison (10.0%) unrepresented while `if/then` (0.1% of sentences) holds 21% of the inventory. Geometry participates in exactly two cognitive mechanisms — salience→attention (bypassed by every licensed lane) and `relation_compiler`'s Hamiltonian ground-state solver (off-serving, fed by a 1.0% math reader) — both stranded behind enumerative front ends that cannot generalize by construction, which is why capability lifts have arrived only as overfit slivers. **Not packs** (0.67%), **not wiring** (failures precede every flag). Full findings F-A…F-E and the remediation plan: `docs/plans/2026-07-28-perception-arc.md`. **Authority:** the Phase-1 pre-registered experiment (H2, GO/NO-GO) + an ADR-0252 amendment (Shay). + - **G-23 · The domain↔pack binding is stated twice and checked in one direction** *(new, 2026-07-28; **pinned** the same day)* — `core/capability/domains.py`'s `DOMAIN_PACKS` declares domain→pack membership for the capability ledger; each `packs/data//manifest.json` independently declares `domain_id`. `domain_contract_predicates.py` **P3** validates `domain_id → a known ledger domain` and is the only binding check that existed. Nothing validated the reverse, and **P3 passes vacuously on a manifest with no `domain_id` at all** — the same shape as its own `test_pack_without_contract_reports_absent`. **Measured: 7 of 9 bound, 0 contradictory, 2 absent** — `en_core_cognition_v1` and `en_core_meta_v1`, both placed in `philosophy_theology`, which is one of the four bands queued to earn a SERVE license behind R-8. The domain whose license is next to be earned is the one whose binding exists in only one place. *Why nothing caught it:* `tests/test_domain_contract_predicates.py` exercises P1–P9 entirely against synthetic `tmp_path` manifests — never against the real nine — and is itself in **no curated suite**. A predicate proven correct on fabricated input and never run on the real input is this arc's defect class stated precisely: its success state and *"it never ran here"* are indistinguishable. diff --git a/docs/assessment/50-execution-plan.md b/docs/assessment/50-execution-plan.md index 6b4954b2..2bb118dc 100644 --- a/docs/assessment/50-execution-plan.md +++ b/docs/assessment/50-execution-plan.md @@ -480,6 +480,8 @@ The arc closes when: **What does *not* close with this arc:** the widening program (G-3) and the chooser (G-4). Those are the next arc, and they should be entered with the registers clean, the paradigm decided, and the evidence loops turning — which is precisely what this plan buys. +**Post-arc addendum (2026-07-28, at `0e1be8be`):** the next arc is now specified and G-3's original formulation is **superseded**. The arc-closing diagnosis (G-24) found the perception layer implements the inversion of ADR-0252 §2 stage 1 — surface-slot templates over the wrong relation set, reading 1.28% of the held-out corpus — and that no served byte is geometric. Widening the template inventory is the one move guaranteed to produce only overfit slivers. The successor plan, with measured findings F-A…F-E, a pre-registered GO/NO-GO experiment on the untested operator-composition branch, and the deletion wave: **`docs/plans/2026-07-28-perception-arc.md`**. + --- *Sequenced per `docs/conceptualizing_engineering_mastery.md`: scrub, delete, simplify, accelerate, automate last. Nothing above optimizes a thing that Wave 1 might delete, and nothing above automates a loop that Wave 3 has not proven turns. Nine corrections in §0 were found by reading code, workflows, branch tips, and ADR supersession banners — none by re-reading the assessment.* diff --git a/docs/plans/2026-07-28-perception-arc.md b/docs/plans/2026-07-28-perception-arc.md new file mode 100644 index 00000000..0af89347 --- /dev/null +++ b/docs/plans/2026-07-28-perception-arc.md @@ -0,0 +1,127 @@ +# The Perception Arc — why capability has not moved, and the plan that moves it + +**Author:** Fable 5 · 2026-07-28 · measured at `0e1be8be` +**Supersedes:** Track B's "widen from 19 constructions" (G-3, as previously specified) — see §4. +**Governed by:** `AGENTS.md` standing philosophy; the pre-registration discipline that produced the §5 NO-GO. + +--- + +## 0. The question this answers + +Everything is implemented. The gate is 4 suites / 1,441 tests, `wrong=0` holds everywhere it is claimed, twelve rulings executed, every register clean. **And capability has not moved.** The reader decides ~1% of the held-out corpus, four deduction bands hold the only earned licences, and nothing else serves. Why? + +This document answers with measurements, not opinion. Every claim below was verified against the source at `0e1be8be` on 2026-07-28. + +--- + +## 1. The diagnosis, in five measured findings + +### F-A · The serving path is 100% symbolic — the geometry never touches it + +| Serving-path module | Lines | Geometric refs | +|---|---|---| +| `generate/meaning_graph/reader.py` | 490 | **0** | +| `generate/meaning_graph/model.py` | 229 | **0** | +| `generate/proof_chain/` (15 files) | 3,916 | **0** | +| `chat/deduction_surface.py` | 298 | **0** | +| `chat/curriculum_surface.py` | 406 | **0** | + +Comprehension is a hand-written template parser (1 regex, 35 string ops) feeding an ROBDD boolean entailment engine. That is classical symbolic AI, and it is the source of **all** served capability. The Cl(4,1) substrate — 76 modules, concentrated in `core/physics` (24) — runs identity continuity, persistence, and the soak invariants. The two systems intersect on the serving path in exactly **one** place (F-D), and the licensed lanes bypass it. + +### F-B · The reader inverts its own ratified design — this is the root cause + +**ADR-0252 §2 stage 1** (ratified 2026-07-19) prescribes, verbatim: + +> *"Reduce surface text to the primitives the substrate represents: quantities (the Number core-system), entities (the Object core-system), and their **relational roles — containment / transfer / accumulation / comparison** (two-argument relational predicates). **Not surface-slot regexes.**"* + +**What was built:** surface-slot templates over **categorical / member / ordering / propositional** relations — Aristotelian syllogistic plus propositional logic. The banned mechanism, over the wrong relation set. + +The corpus measurement confirms the inversion independently. On `holdout_dev/v1` (500 problems → 1,798 sentences): + +| Feature in real English | Share | Reader constructions for it | +|---|---|---| +| numerals (quantity) | 64.7% | — | +| coordination / relative clause | 48.3% | — | +| `how many` questions | 24.4% | 0 | +| possession (`has/have`) | 18.4% | **0** | +| transfer verbs (`bought`, `gave`, `ate`) | 14.6% | **0** | +| quantitative comparison (`twice as many`) | 10.0% | 2 (ordering only) | +| explicit `if/then` | **0.1%** | **4 — 21% of the inventory** | + +**Read rate: 1.28%** (23 of 1,798 sentences; 93.16% fail as `no_template_match` — upstream of vocabulary, upstream of every flag). The relation kinds the ADR prescribes are precisely the ones missing. Not packs (0.67%; `unknown_morphology` never fires). Not wiring (the failure is before any flag is consulted). **Design-to-implementation inversion.** + +### F-C · The template architecture cannot produce chunk lifts — only overfit slivers + +An enumerative parser generalizes by exactly zero: each new construction is a hand-fitted rule matching one surface shape. 19 constructions → 1.28%. Reaching 50% coverage this way means hundreds of hand rules, each an overfit. **Chunk-sized lifts are architecturally unavailable from this front end** — which is why none have appeared despite everything downstream working. + +### F-D · Geometry participates in exactly two cognitive mechanisms — and both are stranded + +1. **Attention.** `use_salience` (default ON) computes field curvature (`SalienceOperator`) → `AttentionOperator` → allowed vocabulary indices, on the generic pipeline lane. But `deduction_surface` and `curriculum_surface` **intercept before generic dispatch** — so the only lanes that ever earned licences bypass the one serving-path mechanism the geometry has. (Also: `use_salience` is one of the two default-ON flags with no recorded rationale — flag register §1.) +2. **The relation compiler.** `core/physics/relation_compiler.py` compiles an affine relation into a `ProblemHamiltonian` whose **ground state is the answer as a null point** — deliberately never evaluating `scale·input + offset` in Python. This is real geometric computation: a relation encoded as an **operator acting on state**. It is off-serving, Tier-1-only (single affine relation), and fed by a math reader that decides **5 of 500** holdout cases (G-21). + +**Both readers — logic (1.28%) and math (1.0%) — are enumerative front ends starving engines that work.** + +### F-E · The §5 NO-GO killed the alignment bridge; the composition bridge was never tested + +§5 refuted structure-as-**point-positions** aligned by conformal Procrustes **under similarity**: the quotient that delivers attribute-invariance annihilates structural contrast. Its own scope note excludes other Cl(4,1) representations — and the untested branch is the one the relation compiler already embodies in miniature: **relations as operators, meaning as composition.** A transfer is a versor acting on a state, not a point pair. Composition ("what operator sequence produces this state?") lives in a different quotient than alignment ("do these configurations match?"), and the §5 argument does not reach it. + +**Summary sentence:** the engines work, the substrate works, the discipline works; the perception layer violates its own ratified design, feeds <2% of real input to the engines, and cannot generalize by construction. Fix perception once, compositionally, per the design that was already ratified — and every downstream engine's capability multiplies instead of crawling. + +--- + +## 2. The plan + +Sequenced per `docs/conceptualizing_engineering_mastery.md`: measure → decide the paradigm → build once → reconnect → delete. Nothing is built before its paradigm question is answered; nothing survives that the new layer supersedes. + +### Phase 0 · Pin the ground truth · **S** + +- **G-24** in the gap register: findings F-A…F-E, Tier A (it re-scopes Track B and Track C's entry conditions). +- **The read-rate lane** (discharges G-21's open ask): `evals/perception/read_rate.py` measuring sentence-level read rate on `holdout_dev/v1` — logic reader and math reader, refusal taxonomy included. Pinned as a **floor ratchet** (1.28% / 1.0% at `0e1be8be`): the number may only rise, and a rise is a reviewed decision. This is the metric the arc is accountable to — capability, not machinery. + +### Phase 1 · The §2 experiment — compositional core-primitive reading · **M** · *pre-registered, GO/NO-GO* + +The §5-shaped experiment for the untested branch. **Hypothesis H2:** a reader that (a) maps text onto the **four ratified primitives** — containment, transfer, accumulation, comparison — plus the Number and Object core-systems; (b) encodes each relation as an **operator** (the `relation_compiler` pattern, generalized); and (c) derives sentence meaning by **composition** of operators — generalizes **multiplicatively**: built from *k* atomic constructions, it reads novel compositions it was never shown. + +**Pre-registered criterion, committed before the run** (same discipline as `299c92be`): +- **Coverage:** ≥ an order of magnitude over the enumerative ceiling on held-out `holdout_dev/v1` sentences — target ≥ 30% sentence read-rate from ≤ 30 declared primitives (vs 19 templates → 1.28%). +- **Honesty:** zero wrong readings. A mis-read is strictly worse than a refusal (G-2's whole lesson); every reading is checked against a gold role-assignment on a labelled slice. +- **Compositionality proof:** held-out *compositions* (novel primitive sequences) must read correctly with **zero per-composition rules** — the assertion that separates a compositional mechanism from a bigger template pile. + +**NO-GO branch, declared now:** if composition over geometric operators does not generalize, the honest conclusion is that Cl(4,1) is a state substrate, not a comprehension mechanism — and the reader is rebuilt as a principled **symbolic compositional grammar** (small combinator core + declared lexicon) instead. Either verdict exits the template treadmill; a well-controlled NO-GO is full credit, as §5 proved. + +### Phase 2 · Build the perception layer per the winner · **L** + +One front end replacing both readers, structured as the repo's own governing pattern (ADR-0263 rule 5 — declared in the table, not the call site): + +- **A small combination core** (code): role binding, operator composition, quantity/entity tracking. This is the only part that is code, and it does not grow with coverage. +- **A construction lexicon** (data): constructions and lexical mappings as declared tables. **Growth = adding rows, not writing parsers.** The gate cost of new coverage is a table diff, reviewable in one glance — the anti-overfit property, made structural. +- Wired to both engines: role-predicate graphs → `proof_chain` (logic); quantity relations → `relation_compiler` corridor (math). `quantity_kernel` (Number) and entity tracking (Object) as the two core-systems the ADR names. + +### Phase 3 · Reconnect the corridor to serving · **M** + +`parse_and_solve` → Hamiltonian corridor is off-serving today. When the new perception layer feeds it, it goes through the **now-honest** licence machinery (post-R-13/R-8: distinct evidence, split outcome mix, `wrong=0`-or-refuse). Capability target measured on the pinned lane: holdout decision rate 1% → tracked leaps as primitive coverage composes. Lifts arrive in chunks or the paradigm is wrong — that is the falsifiable commitment. + +### Phase 4 · The deletion wave · **S** — *the "scrap what earned scrapping" half* + +- **G-3's template-widening program: superseded.** No more hand constructions in the enumerative reader; it stays serving for the logic bands it already handles until Phase 2 supersedes it in place (nothing torn out before its replacement proves out — §6's own discipline, applied honestly this time). +- **Curriculum-serve machinery: parked.** It licenses nothing (R-8 measured: zero bands), and its earnable capability is *declining to answer*. The corpus is kept; the lane sleeps until the new layer gives it something to license. +- **`sensorium/` stays parked** behind P-3's criterion, which is now honest: no modality enters while text perception is at 1%. +- The 15 gate-unreachable suites get re-examined as lanes consolidate — deletion first, promotion second, per philosophy 1. + +### Phase 5 · Governance · **S** — *Shay's, prepared to cost one word each* + +- **ADR-0252 amendment:** record that §2 stage 1 was implemented in inversion of its own text (banned mechanism, wrong primitives), and bind the §6 build clause to Phase 1's pre-registered verdict. +- **`use_salience` ruling:** it is default-ON with no recorded rationale and the licensed lanes bypass it. Phase 2 makes attention either load-bearing on the new lane (keep, document, measure) or decorative (off). The flag register's "no criterion recorded" entry gets its criterion. +- **Ratchet:** the read-rate floor may only rise; every phase reports against it. + +--- + +## 3. What this plan deliberately does not do + +- It does not touch `core/physics`, the algebra, the reliability gate, or the ledger discipline. Those are correct and load-bearing. +- It does not delete the enumerative reader before its replacement proves out — supersession-in-place, per the ADR's own preserved rule. +- It does not promise the geometric branch wins Phase 1. It promises the question is **decided** the way §5 was: pre-registered, cheap, and honest either way. + +## 4. Relationship to the executed assessment arc + +The 2026-07-28 arc (30 commits, `797ebad5..0e1be8be`) built the instruments this plan depends on: honest licences (R-13, R-8), the flag register, the reachability ratchets, the pre-registration precedent, and the eleven standing philosophies. None of it moved capability — and F-C explains why nothing could have: the arc hardened a system whose perception layer cannot generalize. This plan is the capability half the enforcement half was for.