docs(analysis): Phase 2 general-comprehension organ scope
Scope-before-build for the AGI-roadmap make-or-break phase (COMPREHEND). Honest substrate map: the reasoners (binding-graph/proof_chain/grammars) and the articulation path (PropositionGraph) are built, but no organ turns general NL into reasoning-ready structure. Recommends Option B (a general MeaningGraph the existing structures project into), syntax-keyed parse-or-refuse reader, cross-domain proof on the Phase-1 yardstick, and the overfit-trap guardrails. No code.
This commit is contained in:
Shay
parent
ea503f6d51
commit
d97be49ffa
1 changed files with 326 additions and 0 deletions
|
|
@ -0,0 +1,326 @@
|
|||
# Phase 2 — The General Comprehension Organ (scope, not build)
|
||||
|
||||
**Status:** SCOPE — no code. This is the scope-before-build for the make-or-break
|
||||
phase of the AGI-candidacy roadmap
|
||||
([AGI-candidacy-autonomous-improvement-roadmap-2026-06-05.md](./AGI-candidacy-autonomous-improvement-roadmap-2026-06-05.md)).
|
||||
Phase 1 (MEASURE — the cross-domain capability index) is landed (#575); this
|
||||
document scopes Phase 2 (COMPREHEND) so the first increment can be built TDD
|
||||
without falling into the per-domain-matcher overfit trap that would fake the
|
||||
capability number.
|
||||
|
||||
**Reviewer note:** I am the eyes on the implementation; the architectural
|
||||
decision in §4 is presented with a recommendation for design review. Nothing
|
||||
here is committed beyond this document.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why this phase is the gate
|
||||
|
||||
The roadmap's loop is `COMPREHEND → REALIZE → REASON/GROUND/RECALL → RESPOND
|
||||
(assert/estimate/refuse) → PROPOSE → HITL → ACCUMULATE → measurably more
|
||||
capable → repeat`. Every later phase consumes the output of COMPREHEND. If
|
||||
comprehension is narrow, the whole organism is narrow no matter how good the
|
||||
reasoners are. This is exactly what the GSM8K serving numbers were telling us:
|
||||
~92% **refused** is not a reasoning failure, it is a **comprehension** failure —
|
||||
the engine could not turn the prose into a structure the reasoner could touch.
|
||||
|
||||
**The one-line gap:** CORE can *reason* over structured input in several
|
||||
domains, and it can *articulate* a response from field resonance, but it has
|
||||
**no organ that turns arbitrary natural language into reasoning-ready
|
||||
structure.** The reasoning side and the articulation side never connect.
|
||||
|
||||
---
|
||||
|
||||
## 2. The honest substrate map
|
||||
|
||||
### 2a. The articulation side (chat) — shallow comprehension, aimed at responding
|
||||
|
||||
| Module | What it actually does |
|
||||
| --- | --- |
|
||||
| `generate/proposition.py` :: `PropositionGraph` | "prompt and field form a relation blade; a frame is selected by exact CGA inner product against that relation; vocabulary points instantiate the frame slots." Frame-fill from field resonance. |
|
||||
| `generate/graph_planner.py` | `PropositionGraph → ArticulationTarget` (topological walk → ordered articulation steps). This is the **output** path. |
|
||||
| `generate/intent.py`, `generate/realizer.py` | intent classification + deterministic surface realization. |
|
||||
|
||||
This path is **general over input** but **shallow**: it selects a response
|
||||
frame by field resonance and fills slots from vocabulary. It is built to
|
||||
*respond*, not to *understand-for-reasoning*. Its `PropositionGraph` is **not**
|
||||
a refusal-first, reasoning-ready meaning structure — and it shares a name with
|
||||
the logic-side proposition representation (§2b), which we must not conflate.
|
||||
|
||||
### 2b. The reasoning side — strong reasoners, but they consume *structured* input
|
||||
|
||||
| Lane / module | Input it consumes | Meaning structure |
|
||||
| --- | --- | --- |
|
||||
| `evals/deductive_logic` (ROBDD) | **already-formal** facts + rules + query (JSON) | proof_chain proposition repr (ADR-0201/0202), canonicalizer, proof-graph-builder (ADR-0204), modus-ponens (ADR-0205) |
|
||||
| `evals/relational_metric` (`generate/relational_field_reader`) | **narrow templated text** | tiny quantitative grammar: `fact / more_than / fewer_than / sum_of` |
|
||||
| `evals/dimensional` | structured cases | unit/dimension analysis |
|
||||
| GSM8K (`generate/derivation`, `generate/binding_graph`) | math word problems → `MathProblemGraph` | **binding-graph** (ADR-0132): the canonized NL↔reasoning interlingua |
|
||||
|
||||
### 2c. The binding-graph interlingua — neutral, but arithmetic-shaped
|
||||
|
||||
`generate/binding_graph/model.py` (ADR-0132) is described as "the typed compiler
|
||||
boundary between natural language and symbolic reasoning," and INV-26 keeps it
|
||||
**neutral** (it imports no engine/benchmark/domain code). That discipline is
|
||||
exactly right. But:
|
||||
|
||||
- Its closed vocabularies are **arithmetic**: `SEMANTIC_ROLES = {entity,
|
||||
quantity, rate, duration, count, total, difference, ratio, unknown}`;
|
||||
`QUESTION_FORMS = {count, rate, total, difference, ratio, identity}`.
|
||||
- Its only relational node is `BoundEquation` (`lhs := rhs`, quantitative) and a
|
||||
single-symbol `BoundConstraint` (`predicate` string). **There is no general
|
||||
n-ary relation / predicate node, no class-membership, no quantifier.**
|
||||
- Its **only producer** is `generate/binding_graph/adapter.py`, which translates
|
||||
an *already-structured* `MathProblemGraph` — **not raw NL** — into the graph.
|
||||
|
||||
So the "interlingua" exists and is well-disciplined, but today it is the
|
||||
**arithmetic word-problem** interlingua, fed by the math reader, never by a
|
||||
general parser.
|
||||
|
||||
### 2d. The precise gap (the "missing middle")
|
||||
|
||||
```
|
||||
arbitrary NL prose
|
||||
│
|
||||
▼
|
||||
┌──────────────────────┐
|
||||
│ ??? GENERAL │ ← Phase 2: this organ does not exist
|
||||
│ COMPREHENSION ORGAN │
|
||||
└──────────────────────┘
|
||||
│ (general meaning structure)
|
||||
┌──────────┴───────────┐
|
||||
▼ ▼ ▼
|
||||
binding-graph proof_chain relational/dimensional
|
||||
(quantities) propositions grammars
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
the reasoners (already built, already independent-gold)
|
||||
```
|
||||
|
||||
The reasoners are built. The yardstick is built. The articulation path is built.
|
||||
**The general parser from prose into the reasoners' world is the unbuilt
|
||||
make-or-break.**
|
||||
|
||||
---
|
||||
|
||||
## 3. Definitions made precise (carrying the corrected epistemic frame)
|
||||
|
||||
These align with the roadmap's epistemic foundation (honesty designed,
|
||||
estimation learned) and the user's corrections.
|
||||
|
||||
- **Comprehend** = turn arbitrary input into a **structured meaning** keyed on
|
||||
general structure (syntax + grounding), not on domain word-lists. Output is a
|
||||
general meaning structure (§4), or a **refusal** — never a fabricated parse.
|
||||
- **Realize** = integrate that meaning into the held self with an
|
||||
**EpistemicStatus** (told / coherent-with-evidence / verified). "Being told"
|
||||
is first-class: most knowledge arrives as told facts the engine realizes and
|
||||
earns the why/how over time. Realization is what makes intake recallable.
|
||||
- **Intake** is first-class (NOT "no ingestion"): take in inputs, comprehend,
|
||||
realize as structured grounded memory it can recall. The ban is on **bulk
|
||||
indiscriminate absorption** into a database, not on ingesting knowledge.
|
||||
- **Parse-or-refuse floor** = a statement comprehends iff its **structure** maps
|
||||
to the meaning structure via general rules **and** its **content** grounds
|
||||
(known lemmas, or honest typed unknowns). Anything else → refuse. This is how
|
||||
`wrong=0` holds *at the comprehension layer*: the engine never invents a
|
||||
reading it cannot ground. (Note: `wrong=0` here is the comprehension gear of
|
||||
the roadmap's "honesty designed in," not a universal law.)
|
||||
|
||||
**Non-goal restated:** comprehension does **not** include a guess organ. If the
|
||||
structure or grounding is absent, it refuses. Estimation, where it ever applies,
|
||||
is a *learned, ratified* competence built later (Phase 6), never designed into
|
||||
the parser.
|
||||
|
||||
---
|
||||
|
||||
## 4. The architectural decision: what does comprehension emit?
|
||||
|
||||
This is the load-bearing decision and the reason to scope before building.
|
||||
Meaning-structure today is spread across three substrates, none general:
|
||||
|
||||
1. **binding-graph** (quantities/equations) — math-shaped, INV-26 neutral.
|
||||
2. **proof_chain propositions** (ADR-0201/0202) — logic-shaped.
|
||||
3. **PropositionGraph** (`generate/proposition.py`) — field-resonance
|
||||
articulation, wrong tool, name-collision hazard.
|
||||
|
||||
To comprehend general declarative/interrogative prose across domains we need to
|
||||
represent at least: **entities** (have), **n-ary named relations / predicates**
|
||||
(*missing*), **class-membership / subsumption** (*missing*), **attribution /
|
||||
properties** (*missing*), **quantified statements** (*missing*), **quantities &
|
||||
equations** (have, in binding-graph), **logical connectives** (have, in
|
||||
proof_chain).
|
||||
|
||||
### The options
|
||||
|
||||
- **Option A — extend the binding-graph's closed vocab** with general roles +
|
||||
an n-ary relation node + class/quantifier nodes.
|
||||
*Pro:* one canonized neutral meeting point; reuse refusal/provenance/canonical
|
||||
discipline. *Con:* bloats a structure designed for quantities into a
|
||||
god-structure; the closed-vocab ADRs explicitly say "extend deliberately in a
|
||||
future ADR"; couples logic/relations into the math interlingua.
|
||||
|
||||
- **Option B (recommended) — a general meaning structure that the existing
|
||||
structures *project into*.** Comprehension emits a **general claim/meaning
|
||||
graph** (working name `MeaningGraph`) sharing the binding-graph's discipline
|
||||
(frozen/slots, `SourceSpanLink` provenance, refusal-first, `to_canonical_string`,
|
||||
INV-neutral). A thin **projector** maps it to whichever reasoner's input shape
|
||||
(binding-graph for arithmetic, proof_chain propositions for logic, the
|
||||
relational grammar for relational_metric). The binding-graph and proof_chain
|
||||
propositions become **downstream projections**, not rivals.
|
||||
*Pro:* doesn't bloat math; keeps INV-26 (a neutral meeting point); reuses every
|
||||
existing reasoner unchanged; the general organ has one general target. *Con:* a
|
||||
new substrate to design — mitigated by building it **minimally, one class at a
|
||||
time**, per the defer-substrate-vocab discipline.
|
||||
|
||||
- **Option C — reuse `PropositionGraph`.** Rejected: it is field-resonance
|
||||
frame-fill for articulation, not refusal-first reasoning-ready structure.
|
||||
|
||||
### Recommendation
|
||||
|
||||
**Option B, built minimally and use-case-driven.** Do **not** pre-lock a full
|
||||
general vocabulary (that violates the defer-substrate-vocab discipline). Instead:
|
||||
|
||||
> Introduce `MeaningGraph` carrying exactly the node kinds the **first increment**
|
||||
> needs (entities + one general relation kind), with the binding-graph's
|
||||
> refusal/provenance/canonical discipline and an INV firewall keeping it neutral.
|
||||
> Every later class (subsumption, quantifier, attribution) is a deliberate,
|
||||
> use-case-driven vocabulary extension with its own cross-domain proof.
|
||||
|
||||
The **field as a standing hand** (CL(4,1) inner product / incidence) is a
|
||||
*candidate* for relation-consistency checks (transitivity, contradiction) — note
|
||||
it, do **not** depend on it. The field-reasoner wedge found metric
|
||||
reading-independence unproven and field-as-reasoner deferred; comprehension must
|
||||
stand without it, and may later borrow it where it is geometrically honest.
|
||||
|
||||
---
|
||||
|
||||
## 5. The general reader architecture (how the organ works)
|
||||
|
||||
```
|
||||
NL statement
|
||||
│
|
||||
▼ (1) STRUCTURAL PARSE — keyed on SYNTAX, not domain words
|
||||
en_core_syntax_v1 (definitional layer, reviewed) supplies the
|
||||
general structural skeleton: subject–predicate–object, modifiers,
|
||||
quantifiers, connectives. Domain-agnostic.
|
||||
│
|
||||
▼ (2) GROUNDING-FILL — content fills the skeleton
|
||||
entities/relations resolved against packs + vault (known lemmas) or
|
||||
marked as honest typed unknowns. Content is NEVER hard-coded per domain.
|
||||
│
|
||||
▼ (3) PARSE-OR-REFUSE GATE
|
||||
emit MeaningGraph iff structure maps via general rules AND content
|
||||
grounds; otherwise REFUSE (typed, audited). No fabricated reading.
|
||||
│
|
||||
▼ MeaningGraph ──projector──▶ reasoner input (binding-graph / proposition / grammar)
|
||||
```
|
||||
|
||||
The decisive design commitment: **step (1) keys on structure (syntax), step (2)
|
||||
on grounding (packs/vault).** A class of statement comprehends because of its
|
||||
*grammar*, which is domain-agnostic; the *content* that fills it varies by
|
||||
domain. This is what makes the organ general rather than a pile of recognizers —
|
||||
and it is exactly the property the overfit trap violates.
|
||||
|
||||
---
|
||||
|
||||
## 6. Cross-domain proof obligation & overfit-trap guardrails
|
||||
|
||||
The overfit trap: build a per-domain matcher that lifts one lane's coverage and
|
||||
**fakes** the capability number. The Phase-1 yardstick was built precisely to
|
||||
make this visible (geomean → 0 if any domain stays at zero), but the discipline
|
||||
must be enforced at the comprehension layer too:
|
||||
|
||||
1. **Every comprehension *class* is proven on ≥3 distinct domains** with the
|
||||
*same grammar, different content* (e.g. binary relation "X R Y" over kinship,
|
||||
biology, geometry). Works in only one domain ⇒ it is a matcher ⇒ rejected.
|
||||
2. **The Phase-1 capability index is the acceptance gate.** A comprehension
|
||||
increment is accepted only if `breadth` rises (or coverage rises across
|
||||
*multiple* domains' geomean), with `wrong_total == 0`. A one-domain bump that
|
||||
leaves the geomean flat is, by construction, not progress.
|
||||
3. **Schema-defined proof obligation (CLAUDE.md rule).** The parse-or-refuse
|
||||
gate is load-bearing only if a test **meaningfully fails** when a fabricated
|
||||
reading is admitted. Each class ships with a refusal test that fails if the
|
||||
gate is loosened to admit an ungrounded parse.
|
||||
4. **INV firewall for `MeaningGraph` neutrality** (sibling of INV-26): the
|
||||
structure imports no engine/benchmark/domain code, so two independent
|
||||
decodings can meet there honestly.
|
||||
5. **No silent caps.** If an increment bounds coverage (clause types handled,
|
||||
grounding sources), it is logged — silent truncation reads as "general" when
|
||||
it is not.
|
||||
|
||||
---
|
||||
|
||||
## 7. Increment decomposition (build order)
|
||||
|
||||
Each increment is a small, load-bearing PR with the yardstick as its gate.
|
||||
|
||||
### 2a — `MeaningGraph` substrate + the first general class (binary relations), end-to-end
|
||||
- `MeaningGraph` data model (frozen/slots, `SourceSpanLink` provenance,
|
||||
refusal-first, `to_canonical_string`, INV firewall) carrying **entities + one
|
||||
general n-ary relation node**.
|
||||
- The structural reader (step 1–3) for **binary-relation declaratives** ("X R Y"),
|
||||
keyed on syntax via `en_core_syntax_v1`, grounded against packs/vault,
|
||||
parse-or-refuse.
|
||||
- A projector `MeaningGraph → relational grammar` so the existing
|
||||
`relational_metric` reasoner consumes it unchanged (proves the projection
|
||||
pattern on a real reasoner).
|
||||
- **Acceptance:** binary-relation comprehension proven on **≥3 distinct domains**
|
||||
on the capability index; `wrong_total == 0`; refusal tests bite; index digest
|
||||
recorded as the new baseline.
|
||||
|
||||
### 2b — widen relation classes (use-case-driven)
|
||||
- Add class-membership / subsumption ("a raven is a bird", "all ravens are
|
||||
birds") with a projector into the proof_chain proposition repr so the
|
||||
**deductive_logic** reasoner consumes comprehended prose (closing the
|
||||
formal-input gap for the largest lane).
|
||||
- Each class: ≥3-domain proof, wrong=0, refusal test, geomean must move.
|
||||
|
||||
### 2c — attribution / quantity bridges + loop-until-coverage
|
||||
- Attribution ("the ball is red"), and the quantity bridge `MeaningGraph →
|
||||
binding-graph` so comprehended math prose reaches the GSM8K reasoner without
|
||||
the `MathProblemGraph` shortcut.
|
||||
- Loop one class at a time until coverage stops rising (loop-until-dry), each
|
||||
cross-domain-proven.
|
||||
|
||||
---
|
||||
|
||||
## 8. Risks, invariants, non-goals
|
||||
|
||||
**Invariants preserved:** `versor_condition < 1e-6` (untouched — comprehension is
|
||||
symbolic/structural, no field repair); exact CGA recall (no approximate match
|
||||
introduced); `wrong=0` as the comprehension gear (parse-or-refuse); INV-26-style
|
||||
neutrality extended to `MeaningGraph`; reviewed learning stays HITL (comprehension
|
||||
feeds REALIZE/PROPOSE, never self-ratifies).
|
||||
|
||||
**Risks:**
|
||||
- *Vocabulary creep* → mitigated by use-case-driven extension, one class at a
|
||||
time, each with its own ADR and proof (defer-substrate-vocab discipline).
|
||||
- *`MeaningGraph` becoming a third orphan structure* → mitigated by the projector
|
||||
pattern: it must feed an existing reasoner from increment 2a, or it is not
|
||||
built.
|
||||
- *Syntax-pack depth unknown* → `en_core_syntax_v1` is a lexicon/gloss pack, not
|
||||
a parser. 2a must establish how much general structural parsing it actually
|
||||
supports; if insufficient, the first sub-task is a minimal, collision-audited
|
||||
structural-parse capability (NOT a bulk grammar import — recall the #503 syntax
|
||||
revert).
|
||||
- *Name collision* `PropositionGraph` (articulation) vs proof_chain propositions
|
||||
vs `MeaningGraph` → documented here; keep them distinct.
|
||||
|
||||
**Non-goals (this phase):** a guess/estimate organ (Phase 6, learned+ratified);
|
||||
field-as-reasoner (deferred research); bulk corpus ingestion; touching the
|
||||
serving GSM8K metric (this is additive — comprehension feeds reasoners, it does
|
||||
not change their gold).
|
||||
|
||||
---
|
||||
|
||||
## 9. Open questions for design review
|
||||
|
||||
1. **Option B vs A** — do we accept a sibling `MeaningGraph` that existing
|
||||
structures project into, or extend the binding-graph in place? (Recommend B.)
|
||||
2. **First class** — binary relations as 2a's first class, projecting into
|
||||
`relational_metric`? (Recommend yes: smallest general structure beyond
|
||||
entities, with an existing reasoner + reader to lift from.)
|
||||
3. **Syntax-parse depth** — is `en_core_syntax_v1` enough for general structural
|
||||
parsing of declaratives, or does 2a need a minimal structural-parse capability
|
||||
first? (Needs a spike inside 2a before the reader is built.)
|
||||
4. **Field standing-hand** — reserve CL(4,1) incidence as the relation-consistency
|
||||
checker for a later increment, or leave it out entirely until the wedge
|
||||
resolves? (Recommend: note, don't depend.)
|
||||
Loading…
Reference in a new issue