docs(plans,assessment): G-24 — the perception diagnosis, and the Perception Arc plan
Why capability has not moved despite everything being implemented, answered by
measurement rather than opinion, at 0e1be8be:
F-A The serving path is 100% symbolic. reader.py (490), proof_chain (3,916),
both surface composers: zero geometric references. All served capability
comes from a template parser feeding an ROBDD engine.
F-B The reader INVERTS its own ratified design. ADR-0252 §2 stage 1
prescribes containment/transfer/accumulation/comparison and bans
surface-slot regexes verbatim; the implementation is surface-slot
regexes over categorical/member/ordering/propositional. Corpus agrees:
1.28% read rate, possession/transfer/quantity absent, if/then (0.1% of
sentences) holding 21% of the inventory.
F-C Template architecture cannot produce chunk lifts — each construction is
one hand-fitted rule; generalization is zero by construction. This is
why lifts have only ever arrived as overfit slivers.
F-D Geometry does exactly two cognitive jobs and both are stranded:
salience->attention (bypassed by every licensed lane) and
relation_compiler's Hamiltonian ground-state solver (off-serving,
behind a 1.0% math reader). Real geometric computation exists — it
"NEVER evaluates scale*input+offset in Python" — and nothing feeds it.
F-E The §5 NO-GO killed alignment-on-points-under-similarity. The
composition branch — relations as OPERATORS, meaning as composition,
the pattern relation_compiler already embodies — was never tested.
Plan: docs/plans/2026-07-28-perception-arc.md. Phase 0 pins the 1.28%/1.0%
read-rate floors as the metric the arc is accountable to. Phase 1 is a
§5-shaped pre-registered GO/NO-GO on hypothesis H2 (compositional
core-primitive reading over operator encodings; NO-GO branch declared: a
principled compositional grammar instead — either verdict exits the template
treadmill). Phase 2 builds ONE perception layer as combinator core + declared
construction lexicon, so coverage growth is table rows, not parsers. Phase 3
reconnects the Hamiltonian corridor to serving through the now-honest licence
machinery. Phase 4 deletes what earned it: G-3's template-widening program
superseded, curriculum-serve machinery parked. Phase 5 prepares the ADR-0252
amendment and the use_salience ruling for one-word decisions.
Explicitly untouched: core/physics, the algebra, the reliability gate, the
ledger discipline — correct and load-bearing. The enumerative reader stays
serving until its replacement proves out: supersession-in-place, per the ADR's
own preserved rule.
G-24 registered Tier A. G-3 as previously specified is superseded, recorded in
the execution plan's closing addendum rather than silently.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wcw2pnMBwyvmNyQg4uPEt4
This commit is contained in:
parent
0e1be8be16
commit
a4f567f097
3 changed files with 131 additions and 0 deletions
|
|
@ -201,6 +201,8 @@ Own `use_salience`, the two underived constants, the self-narrowing budget feedb
|
|||
**The mechanism, and why it is G-7's story again:** one commit updated the lane, its verifier and its test, and missed one downstream artifact. The pin that guards that artifact — `tests/test_claims_md_is_current.py` — **exists and is an orphan in no curated suite**, so nothing surfaced it until this arc ran the full tree once. Registered on the gate as part of the fix.
|
||||
- **G-21 · The math reader decides 1.0% of `holdout_dev/v1`** *(new, 2026-07-28)* — measured while building the §5 corpus: `parse_and_solve` returns a selected graph for **5 of 500** held-out cases, all carrying the same relational skeleton (`compare_multiplicative`); on the public lane, 24/150. This is a sharper measurement of the comprehension frontier than G-3's construction count, on the corpus the project already treats as its held-out standard, and it is why ADR-0252 §5.1's four-structure corpus is not extractable today. Distinct from G-3 (which counts *constructions* the general reader admits); this counts *cases decided* on a standing eval corpus. **Authority:** the widening program (G-3) + a ruling on whether holdout decision-rate becomes a tracked lane metric.
|
||||
- **G-20 · The `refusal_reason` materialisation** — typed refusal evidence exists and is discarded at the public `str` boundary; the plumbing for materialisation already landed. Cross-listed as H-3. **Authority:** small ADR (anticipated by the ADR-0024 chain).
|
||||
- **G-24 · The perception layer inverts its own ratified design, and no served byte is geometric** *(new, 2026-07-28 — Tier A; the arc-closing diagnosis)* — Measured at `0e1be8be`: the entire serving path (`meaning_graph/reader.py` 490 lines, `proof_chain/` 3,916, both surface composers) contains **zero** geometric references — comprehension is a template parser feeding an ROBDD engine. **ADR-0252 §2 stage 1 prescribes** *"containment / transfer / accumulation / comparison … **Not surface-slot regexes**"* — the implementation is surface-slot regexes over categorical/member/ordering/propositional. The corpus confirms the inversion independently: on `holdout_dev/v1`, **read rate 1.28%** (93.16% `no_template_match`), with possession (18.4% of sentences), transfer verbs (14.6%) and quantity comparison (10.0%) unrepresented while `if/then` (0.1% of sentences) holds 21% of the inventory. Geometry participates in exactly two cognitive mechanisms — salience→attention (bypassed by every licensed lane) and `relation_compiler`'s Hamiltonian ground-state solver (off-serving, fed by a 1.0% math reader) — both stranded behind enumerative front ends that cannot generalize by construction, which is why capability lifts have arrived only as overfit slivers. **Not packs** (0.67%), **not wiring** (failures precede every flag). Full findings F-A…F-E and the remediation plan: `docs/plans/2026-07-28-perception-arc.md`. **Authority:** the Phase-1 pre-registered experiment (H2, GO/NO-GO) + an ADR-0252 amendment (Shay).
|
||||
|
||||
- **G-23 · The domain↔pack binding is stated twice and checked in one direction** *(new, 2026-07-28; **pinned** the same day)* — `core/capability/domains.py`'s `DOMAIN_PACKS` declares domain→pack membership for the capability ledger; each `packs/data/<pack>/manifest.json` independently declares `domain_id`. `domain_contract_predicates.py` **P3** validates `domain_id → a known ledger domain` and is the only binding check that existed. Nothing validated the reverse, and **P3 passes vacuously on a manifest with no `domain_id` at all** — the same shape as its own `test_pack_without_contract_reports_absent`. **Measured: 7 of 9 bound, 0 contradictory, 2 absent** — `en_core_cognition_v1` and `en_core_meta_v1`, both placed in `philosophy_theology`, which is one of the four bands queued to earn a SERVE license behind R-8. The domain whose license is next to be earned is the one whose binding exists in only one place.
|
||||
|
||||
*Why nothing caught it:* `tests/test_domain_contract_predicates.py` exercises P1–P9 entirely against synthetic `tmp_path` manifests — never against the real nine — and is itself in **no curated suite**. A predicate proven correct on fabricated input and never run on the real input is this arc's defect class stated precisely: its success state and *"it never ran here"* are indistinguishable.
|
||||
|
|
|
|||
|
|
@ -480,6 +480,8 @@ The arc closes when:
|
|||
|
||||
**What does *not* close with this arc:** the widening program (G-3) and the chooser (G-4). Those are the next arc, and they should be entered with the registers clean, the paradigm decided, and the evidence loops turning — which is precisely what this plan buys.
|
||||
|
||||
**Post-arc addendum (2026-07-28, at `0e1be8be`):** the next arc is now specified and G-3's original formulation is **superseded**. The arc-closing diagnosis (G-24) found the perception layer implements the inversion of ADR-0252 §2 stage 1 — surface-slot templates over the wrong relation set, reading 1.28% of the held-out corpus — and that no served byte is geometric. Widening the template inventory is the one move guaranteed to produce only overfit slivers. The successor plan, with measured findings F-A…F-E, a pre-registered GO/NO-GO experiment on the untested operator-composition branch, and the deletion wave: **`docs/plans/2026-07-28-perception-arc.md`**.
|
||||
|
||||
---
|
||||
|
||||
*Sequenced per `docs/conceptualizing_engineering_mastery.md`: scrub, delete, simplify, accelerate, automate last. Nothing above optimizes a thing that Wave 1 might delete, and nothing above automates a loop that Wave 3 has not proven turns. Nine corrections in §0 were found by reading code, workflows, branch tips, and ADR supersession banners — none by re-reading the assessment.*
|
||||
|
|
|
|||
127
docs/plans/2026-07-28-perception-arc.md
Normal file
127
docs/plans/2026-07-28-perception-arc.md
Normal file
|
|
@ -0,0 +1,127 @@
|
|||
# The Perception Arc — why capability has not moved, and the plan that moves it
|
||||
|
||||
**Author:** Fable 5 · 2026-07-28 · measured at `0e1be8be`
|
||||
**Supersedes:** Track B's "widen from 19 constructions" (G-3, as previously specified) — see §4.
|
||||
**Governed by:** `AGENTS.md` standing philosophy; the pre-registration discipline that produced the §5 NO-GO.
|
||||
|
||||
---
|
||||
|
||||
## 0. The question this answers
|
||||
|
||||
Everything is implemented. The gate is 4 suites / 1,441 tests, `wrong=0` holds everywhere it is claimed, twelve rulings executed, every register clean. **And capability has not moved.** The reader decides ~1% of the held-out corpus, four deduction bands hold the only earned licences, and nothing else serves. Why?
|
||||
|
||||
This document answers with measurements, not opinion. Every claim below was verified against the source at `0e1be8be` on 2026-07-28.
|
||||
|
||||
---
|
||||
|
||||
## 1. The diagnosis, in five measured findings
|
||||
|
||||
### F-A · The serving path is 100% symbolic — the geometry never touches it
|
||||
|
||||
| Serving-path module | Lines | Geometric refs |
|
||||
|---|---|---|
|
||||
| `generate/meaning_graph/reader.py` | 490 | **0** |
|
||||
| `generate/meaning_graph/model.py` | 229 | **0** |
|
||||
| `generate/proof_chain/` (15 files) | 3,916 | **0** |
|
||||
| `chat/deduction_surface.py` | 298 | **0** |
|
||||
| `chat/curriculum_surface.py` | 406 | **0** |
|
||||
|
||||
Comprehension is a hand-written template parser (1 regex, 35 string ops) feeding an ROBDD boolean entailment engine. That is classical symbolic AI, and it is the source of **all** served capability. The Cl(4,1) substrate — 76 modules, concentrated in `core/physics` (24) — runs identity continuity, persistence, and the soak invariants. The two systems intersect on the serving path in exactly **one** place (F-D), and the licensed lanes bypass it.
|
||||
|
||||
### F-B · The reader inverts its own ratified design — this is the root cause
|
||||
|
||||
**ADR-0252 §2 stage 1** (ratified 2026-07-19) prescribes, verbatim:
|
||||
|
||||
> *"Reduce surface text to the primitives the substrate represents: quantities (the Number core-system), entities (the Object core-system), and their **relational roles — containment / transfer / accumulation / comparison** (two-argument relational predicates). **Not surface-slot regexes.**"*
|
||||
|
||||
**What was built:** surface-slot templates over **categorical / member / ordering / propositional** relations — Aristotelian syllogistic plus propositional logic. The banned mechanism, over the wrong relation set.
|
||||
|
||||
The corpus measurement confirms the inversion independently. On `holdout_dev/v1` (500 problems → 1,798 sentences):
|
||||
|
||||
| Feature in real English | Share | Reader constructions for it |
|
||||
|---|---|---|
|
||||
| numerals (quantity) | 64.7% | — |
|
||||
| coordination / relative clause | 48.3% | — |
|
||||
| `how many` questions | 24.4% | 0 |
|
||||
| possession (`has/have`) | 18.4% | **0** |
|
||||
| transfer verbs (`bought`, `gave`, `ate`) | 14.6% | **0** |
|
||||
| quantitative comparison (`twice as many`) | 10.0% | 2 (ordering only) |
|
||||
| explicit `if/then` | **0.1%** | **4 — 21% of the inventory** |
|
||||
|
||||
**Read rate: 1.28%** (23 of 1,798 sentences; 93.16% fail as `no_template_match` — upstream of vocabulary, upstream of every flag). The relation kinds the ADR prescribes are precisely the ones missing. Not packs (0.67%; `unknown_morphology` never fires). Not wiring (the failure is before any flag is consulted). **Design-to-implementation inversion.**
|
||||
|
||||
### F-C · The template architecture cannot produce chunk lifts — only overfit slivers
|
||||
|
||||
An enumerative parser generalizes by exactly zero: each new construction is a hand-fitted rule matching one surface shape. 19 constructions → 1.28%. Reaching 50% coverage this way means hundreds of hand rules, each an overfit. **Chunk-sized lifts are architecturally unavailable from this front end** — which is why none have appeared despite everything downstream working.
|
||||
|
||||
### F-D · Geometry participates in exactly two cognitive mechanisms — and both are stranded
|
||||
|
||||
1. **Attention.** `use_salience` (default ON) computes field curvature (`SalienceOperator`) → `AttentionOperator` → allowed vocabulary indices, on the generic pipeline lane. But `deduction_surface` and `curriculum_surface` **intercept before generic dispatch** — so the only lanes that ever earned licences bypass the one serving-path mechanism the geometry has. (Also: `use_salience` is one of the two default-ON flags with no recorded rationale — flag register §1.)
|
||||
2. **The relation compiler.** `core/physics/relation_compiler.py` compiles an affine relation into a `ProblemHamiltonian` whose **ground state is the answer as a null point** — deliberately never evaluating `scale·input + offset` in Python. This is real geometric computation: a relation encoded as an **operator acting on state**. It is off-serving, Tier-1-only (single affine relation), and fed by a math reader that decides **5 of 500** holdout cases (G-21).
|
||||
|
||||
**Both readers — logic (1.28%) and math (1.0%) — are enumerative front ends starving engines that work.**
|
||||
|
||||
### F-E · The §5 NO-GO killed the alignment bridge; the composition bridge was never tested
|
||||
|
||||
§5 refuted structure-as-**point-positions** aligned by conformal Procrustes **under similarity**: the quotient that delivers attribute-invariance annihilates structural contrast. Its own scope note excludes other Cl(4,1) representations — and the untested branch is the one the relation compiler already embodies in miniature: **relations as operators, meaning as composition.** A transfer is a versor acting on a state, not a point pair. Composition ("what operator sequence produces this state?") lives in a different quotient than alignment ("do these configurations match?"), and the §5 argument does not reach it.
|
||||
|
||||
**Summary sentence:** the engines work, the substrate works, the discipline works; the perception layer violates its own ratified design, feeds <2% of real input to the engines, and cannot generalize by construction. Fix perception once, compositionally, per the design that was already ratified — and every downstream engine's capability multiplies instead of crawling.
|
||||
|
||||
---
|
||||
|
||||
## 2. The plan
|
||||
|
||||
Sequenced per `docs/conceptualizing_engineering_mastery.md`: measure → decide the paradigm → build once → reconnect → delete. Nothing is built before its paradigm question is answered; nothing survives that the new layer supersedes.
|
||||
|
||||
### Phase 0 · Pin the ground truth · **S**
|
||||
|
||||
- **G-24** in the gap register: findings F-A…F-E, Tier A (it re-scopes Track B and Track C's entry conditions).
|
||||
- **The read-rate lane** (discharges G-21's open ask): `evals/perception/read_rate.py` measuring sentence-level read rate on `holdout_dev/v1` — logic reader and math reader, refusal taxonomy included. Pinned as a **floor ratchet** (1.28% / 1.0% at `0e1be8be`): the number may only rise, and a rise is a reviewed decision. This is the metric the arc is accountable to — capability, not machinery.
|
||||
|
||||
### Phase 1 · The §2 experiment — compositional core-primitive reading · **M** · *pre-registered, GO/NO-GO*
|
||||
|
||||
The §5-shaped experiment for the untested branch. **Hypothesis H2:** a reader that (a) maps text onto the **four ratified primitives** — containment, transfer, accumulation, comparison — plus the Number and Object core-systems; (b) encodes each relation as an **operator** (the `relation_compiler` pattern, generalized); and (c) derives sentence meaning by **composition** of operators — generalizes **multiplicatively**: built from *k* atomic constructions, it reads novel compositions it was never shown.
|
||||
|
||||
**Pre-registered criterion, committed before the run** (same discipline as `299c92be`):
|
||||
- **Coverage:** ≥ an order of magnitude over the enumerative ceiling on held-out `holdout_dev/v1` sentences — target ≥ 30% sentence read-rate from ≤ 30 declared primitives (vs 19 templates → 1.28%).
|
||||
- **Honesty:** zero wrong readings. A mis-read is strictly worse than a refusal (G-2's whole lesson); every reading is checked against a gold role-assignment on a labelled slice.
|
||||
- **Compositionality proof:** held-out *compositions* (novel primitive sequences) must read correctly with **zero per-composition rules** — the assertion that separates a compositional mechanism from a bigger template pile.
|
||||
|
||||
**NO-GO branch, declared now:** if composition over geometric operators does not generalize, the honest conclusion is that Cl(4,1) is a state substrate, not a comprehension mechanism — and the reader is rebuilt as a principled **symbolic compositional grammar** (small combinator core + declared lexicon) instead. Either verdict exits the template treadmill; a well-controlled NO-GO is full credit, as §5 proved.
|
||||
|
||||
### Phase 2 · Build the perception layer per the winner · **L**
|
||||
|
||||
One front end replacing both readers, structured as the repo's own governing pattern (ADR-0263 rule 5 — declared in the table, not the call site):
|
||||
|
||||
- **A small combination core** (code): role binding, operator composition, quantity/entity tracking. This is the only part that is code, and it does not grow with coverage.
|
||||
- **A construction lexicon** (data): constructions and lexical mappings as declared tables. **Growth = adding rows, not writing parsers.** The gate cost of new coverage is a table diff, reviewable in one glance — the anti-overfit property, made structural.
|
||||
- Wired to both engines: role-predicate graphs → `proof_chain` (logic); quantity relations → `relation_compiler` corridor (math). `quantity_kernel` (Number) and entity tracking (Object) as the two core-systems the ADR names.
|
||||
|
||||
### Phase 3 · Reconnect the corridor to serving · **M**
|
||||
|
||||
`parse_and_solve` → Hamiltonian corridor is off-serving today. When the new perception layer feeds it, it goes through the **now-honest** licence machinery (post-R-13/R-8: distinct evidence, split outcome mix, `wrong=0`-or-refuse). Capability target measured on the pinned lane: holdout decision rate 1% → tracked leaps as primitive coverage composes. Lifts arrive in chunks or the paradigm is wrong — that is the falsifiable commitment.
|
||||
|
||||
### Phase 4 · The deletion wave · **S** — *the "scrap what earned scrapping" half*
|
||||
|
||||
- **G-3's template-widening program: superseded.** No more hand constructions in the enumerative reader; it stays serving for the logic bands it already handles until Phase 2 supersedes it in place (nothing torn out before its replacement proves out — §6's own discipline, applied honestly this time).
|
||||
- **Curriculum-serve machinery: parked.** It licenses nothing (R-8 measured: zero bands), and its earnable capability is *declining to answer*. The corpus is kept; the lane sleeps until the new layer gives it something to license.
|
||||
- **`sensorium/` stays parked** behind P-3's criterion, which is now honest: no modality enters while text perception is at 1%.
|
||||
- The 15 gate-unreachable suites get re-examined as lanes consolidate — deletion first, promotion second, per philosophy 1.
|
||||
|
||||
### Phase 5 · Governance · **S** — *Shay's, prepared to cost one word each*
|
||||
|
||||
- **ADR-0252 amendment:** record that §2 stage 1 was implemented in inversion of its own text (banned mechanism, wrong primitives), and bind the §6 build clause to Phase 1's pre-registered verdict.
|
||||
- **`use_salience` ruling:** it is default-ON with no recorded rationale and the licensed lanes bypass it. Phase 2 makes attention either load-bearing on the new lane (keep, document, measure) or decorative (off). The flag register's "no criterion recorded" entry gets its criterion.
|
||||
- **Ratchet:** the read-rate floor may only rise; every phase reports against it.
|
||||
|
||||
---
|
||||
|
||||
## 3. What this plan deliberately does not do
|
||||
|
||||
- It does not touch `core/physics`, the algebra, the reliability gate, or the ledger discipline. Those are correct and load-bearing.
|
||||
- It does not delete the enumerative reader before its replacement proves out — supersession-in-place, per the ADR's own preserved rule.
|
||||
- It does not promise the geometric branch wins Phase 1. It promises the question is **decided** the way §5 was: pre-registered, cheap, and honest either way.
|
||||
|
||||
## 4. Relationship to the executed assessment arc
|
||||
|
||||
The 2026-07-28 arc (30 commits, `797ebad5..0e1be8be`) built the instruments this plan depends on: honest licences (R-13, R-8), the flag register, the reachability ratchets, the pre-registration precedent, and the eleven standing philosophies. None of it moved capability — and F-C explains why nothing could have: the arc hardened a system whose perception layer cannot generalize. This plan is the capability half the enforcement half was for.
|
||||
Loading…
Reference in a new issue