docs(comprehension): whole-system organ capability ledger + fix ADR-0211 number collision

Consolidation/true-up after #646-#648. Records the off-serving comprehension
system as one ledger: R1 relational arithmetic, R2 finite-integer constraints,
R3 explicit single-rate + exact minute/hour conversion, the router/contemplation/
proposal loop, the proposal-review reporter, idle_tick read-only visibility, and
the standing router-organ-hygiene invariant. Pins the whole-system lane state
(answer_wrong==0 everywhere) and the off-serving import-disjointness guarantee.

Also fixes a documentation-integrity defect the R2 batch introduced: the R2 ADR
took number 0211, which collided with ADR-0211 (Conformal Falsification Bench,
2026-06-06 — earlier, test-pinned, depended on by ADR-0216). The conformal ADR
keeps 0211; the R2 ADR is renumbered to the next free number 0217. Ratified
decision content unchanged; only the identifier and references move.
This commit is contained in:
Shay 2026-06-08 06:15:51 -07:00
parent 801f0e23dd
commit df1d14dce4
5 changed files with 172 additions and 5 deletions

View file

@ -48,7 +48,7 @@ admitting any would breach wrong=0. None is a coverage gap.
Two-category finite-integer count/weight systems: read → exact Cramer solve → answer-choice Two-category finite-integer count/weight systems: read → exact Cramer solve → answer-choice
verify (with contradiction flagging). **6 families:** buses/seats, animals/legs, tickets/price, verify (with contradiction flagging). **6 families:** buses/seats, animals/legs, tickets/price,
coins/value, boxes/capacity, vehicles/wheels. Detail: `r2-inventory-ledger-2026-06-07.md`, ADR-0211. coins/value, boxes/capacity, vehicles/wheels. Detail: `r2-inventory-ledger-2026-06-07.md`, ADR-0217.
## 5. R2 remaining refusals ## 5. R2 remaining refusals

View file

@ -0,0 +1,162 @@
# Comprehension organ capability ledger
**As of:** `main` @ `801f0e23` (after #646#648). A take-stock record of the
off-serving comprehension system built so far — what each organ does, where it
lives, the lane that proves it, and what is deliberately deferred.
This is a **consolidation** artifact (docs only, no capability change). It exists
so the next capability lands against a written whole rather than a stack of
per-batch ledgers.
## The shape of the work
We stopped patching one GSM8K shape with one reader rule. Each capability is now
a small **organ** built down a fixed ladder, off the sealed serving path:
```text
gold (ruler) → setup oracle → solver → answer-verifier → reader → router/contemplation → proposal ledger
```
- **gold / setup oracle** — span-free canonical *setup* signature; a fixture's
`expect` (`solved` / `solver_refuses` / `reader_refuses`) is checked against an
independent gold ruler, not against the reader that produced it.
- **solver** — pure typed computation over the setup; refuses (never guesses)
outside its competence.
- **answer-verifier** — ties a solved value to a labeled answer choice
(`generate/answer_choices/verify.py`), shared across organs.
- **reader** — text → typed setup, or a closed-set `Refusal`.
- **router / contemplation** — picks exactly one organ; on its own family a
refusal can become a **proposal**; on foreign text it must step aside.
- **proposal ledger** — proposals are `proposal_only` / `mounted:false` /
`requires_review:true`, content-addressed, never self-installed; they flow
through the existing `teaching/*` HITL flywheel (ADR-0055/56/57), never a
parallel correction path.
### The off-serving guarantee (load-bearing)
Every organ imports **no** `generate.derivation` and **no**
`core.reliability_gate`. That import-disjointness is what makes it
*structurally impossible* for this whole subsystem to regress the sealed GSM8K
serving metric or the pinned-SHA lanes. It is checked, not asserted — the
router-hygiene + off-serving greps are part of each batch's acceptance.
## Organs
### R1 — relational arithmetic
- **Capability.** "Quantity per container × containers", inverse frames, and the
relational single-step arithmetic family. The first organ; established the
gold→oracle→reader ladder.
- **Lives in.** `generate/` reader + `evals/setup_oracle/` (gold `r1_gold.jsonl`).
- **Lane.** setup oracle **7 / 0 / 3** (7 solved, 0 wrong, 3 refused-correct).
- **Ledger.** `docs/analysis/r1-inventory-ledger-*.md`.
### R2 — finite integer constraints
- **Capability.** Two-category integer constraint problems (a finite-integer
constraint compiler: enumerate the bounded integer solution space, refuse if
not uniquely determined). ADR-0217 (renumbered from 0211 — collided with the
conformal-falsification-bench ADR; see that ADR's header).
- **Lives in.** `generate/` constraint reader + `evals/constraint_oracle/`
(gold + reader lane).
- **Lane.** reader **10 / 0 / 0**.
- **Ledger.** `docs/analysis/r2-inventory-ledger-*.md`;
`docs/decisions/ADR-0217-r2-finite-integer-constraint-compiler.md`.
### R3 — explicit single-rate (+ exact minute/hour conversion)
- **Capability.** A single explicit rate (`<N> <plural> per <singular>`), one
duration or quantity unknown, compound-unit algebra
(`mile/hour = quantity/time`). R3.2 adds **exact** time-unit conversion:
`60 miles per hour for 30 minutes → 30 miles`. Conversion is rational
(`fractions.Fraction`), confined to the solver, **int-or-refuse — no float
path**. A non-convertible duration (`…per hour for 3 gallons`) refuses
`rate_unit_mismatch` and stays a proposal surface.
- **Lives in.** `generate/rate_comprehension/`
(`units` / `model` / `conversion` / `solver` / `reader`) +
`evals/rate_oracle/` (gold `rate_gold.jsonl`).
- **Lane.** gold **13 / 13 valid** (7 solved / 2 solver_refuses / 4 reader_refuses);
reader **9 setup_correct / 0 wrong / 4 refused** → answers **7 / 0 / 6**.
- **Ledger.** `docs/analysis/r3-rate-inventory-ledger-*.md`,
`r3-2-unit-conversion-2026-06-08.md`.
## Shared machinery
### Router + contemplation + proposal loop
- **Router** (`core/comprehension_attempt/`) classifies text and routes to
**exactly one** organ (`r1_quantitative` / `r2_constraints` / `r3_rate`),
collecting one attempt per organ and selecting the unique `setup_correct`.
- **Contemplation** (`generate/contemplation/pass_manager.py`) drives
read→solve→verify and lands on one of a closed set of **terminals**
(`SOLVED_VERIFIED` / `REFUSED_KNOWN_BOUNDARY` / `PROPOSAL_EMITTED` / …).
- **Failure families** (`core/comprehension_attempt/failure_family.py`) map each
closed-set refusal reason to a family with `must_remain_refused` /
`proposal_allowed` / `owner`. Only *rate-like growth surfaces*
(`unsupported_rate_duration`, the unsupported system/clause families) propose;
boundaries that must stay refused (non-integer, temporal-state, underdetermined)
do not.
### Proposal-review reporter (RPT)
- `core/proposal_review/` — a read-only reporter over the emitted proposal
ledger: counts, families, owners, content addresses. Deterministic CLI
(`python -m core.proposal_review`). It *reports*; it never mounts or ratifies.
- **Ledger.** `docs/analysis/proposal-review-reporter-*.md`.
### idle_tick read-only visibility (IT)
- `chat/runtime.py::idle_tick` gained an **opt-in, read-only** proposal-review
sub-pass (`RuntimeConfig.review_pending_proposals`, default `False`),
failure-isolated, that surfaces pending proposals during idle without doing
work, setting a checkpoint, or ratifying anything.
- **Ledger.** `docs/analysis/proposal-review-idle-integration-*.md`.
## Standing invariant — router/organ hygiene
> **No organ may block another organ's legitimate proposal unless it has first
> positively recognized the input as its own family.** On foreign text an organ
> must refuse with the non-substantive `input_shape` family — never a
> substantive boundary.
Pinned by `tests/test_router_organ_hygiene.py` (#646): for each organ × every
*other* organ's gold, the refusal must map to `input_shape`. This caught the
recurring over-broad-refusal hazard three times before it was crystallized
(N6 `category_pair_not_found`, R3e `temporal_state`, R3.1 `missing_rate`). Every
new organ must clear this lane before it ships.
- **Ledger.** `docs/analysis/router-organ-hygiene-invariant-*.md`.
## Whole-system lane state (this commit)
| Lane | State |
|---|---|
| R1 setup oracle | 7 / 0 / 3 |
| R2 reader | 10 / 0 / 0 |
| R3 gold | 13 / 13 valid (7 / 2 / 4) |
| R3 reader → answers | 9 / 0 / 4 → 7 / 0 / 6 |
| router-organ-hygiene | green |
| off-serving (import grep) | clean — no `generate.derivation` / `core.reliability_gate` |
| GSM8K serving | unchanged (sealed) |
`answer_wrong == 0` across every organ. The discipline is "6/0/6 over 11/2/0" —
breadth never bought with a single wrong answer.
## Deferred (named, not forgotten)
- **R3.3** — length (`mile ↔ km`) and currency (`dollar ↔ cent`) conversion.
Build only if an existing R3 fixture/proposal needs it; the rational-conversion
substrate is already proven for time.
- **Combined / multi-rate** — two rates composing (work rates, closing speeds).
Next major capability; build down the full ladder
(gold + setup oracle → solver → reader → router/contemplation wiring), **not**
reader-first.
- **Clock-time intervals**, compound conversions, percentage/ratio organs.
## Next capability (decision pending operator go)
Per the standing plan: after this consolidation, either **R3.3** (only if an
existing R3 fixture/proposal demands it) or, preferred, **combined-rate** down
the ladder `CMB-a` (gold + setup oracle) → `CMB-b` (solver) → `CMB-c` (reader) →
`CMB-d` (router/contemplation wiring + ledger). Do **not** start at the
combined-rate reader.

View file

@ -11,7 +11,7 @@ This is the R2 twin of the R1 ledger: a decision artifact recording exactly whic
families the off-serving organ now *reads, solves, and verifies*, which it *refuses*, and the families the off-serving organ now *reads, solves, and verifies*, which it *refuses*, and the
gate protecting each. R2 is disjoint from the GSM8K serving path (imports no gate protecting each. R2 is disjoint from the GSM8K serving path (imports no
`generate.derivation` / `core.reliability_gate`), so none of this moves the sealed serving `generate.derivation` / `core.reliability_gate`), so none of this moves the sealed serving
metric. See ADR-0211 for the contract. metric. See ADR-0217 for the contract (renumbered from 0211 — see that ADR's header).
## Reproduce ## Reproduce
@ -73,7 +73,7 @@ bus / seat chicken / leg ticket / price
coin / value box / capacity vehicle / wheel (+ tool / price) coin / value box / capacity vehicle / wheel (+ tool / price)
``` ```
## Deferred to R3 (NOT in this batch — see ADR-0211 §2) ## Deferred to R3 (NOT in this batch — see ADR-0217 §2)
- ≥3 categories, inequalities, multi-step / mixed constraints, distractor exclusion, - ≥3 categories, inequalities, multi-step / mixed constraints, distractor exclusion,
rates / unit conversion. rates / unit conversion.

View file

@ -1,4 +1,9 @@
# ADR-0211 — R2: Finite-Integer Linear-Constraint Setup Compiler (off-serving) # ADR-0217 — R2: Finite-Integer Linear-Constraint Setup Compiler (off-serving)
> **Renumbered 2026-06-08:** originally landed as ADR-0211, which collided with
> ADR-0211 (Conformal Falsification Bench, 2026-06-06). The conformal ADR is the
> incumbent (earlier, test-pinned, depended on by ADR-0216); this R2 ADR moves to
> the next free number 0217. The ratified decision content is unchanged.
**Status:** Accepted (ratified 2026-06-07) **Status:** Accepted (ratified 2026-06-07)
**Date:** 2026-06-07 **Date:** 2026-06-07

View file

@ -22,7 +22,7 @@ Recognizers and their wrong=0 guards:
else ``query_target_not_a_category``). else ``query_target_not_a_category``).
``too_many_categories`` / ``missing_total_count`` / ``missing_weighted_total`` are the gold's ``too_many_categories`` / ``missing_total_count`` / ``missing_weighted_total`` are the gold's
closed ``reader_reason`` set (ADR-0211). ``coefficient_unit_mismatch`` / ``category_pair_not_found`` closed ``reader_reason`` set (ADR-0217). ``coefficient_unit_mismatch`` / ``category_pair_not_found``
/ ``query_target_not_a_category`` are defensive guards with no gold fixture (tested by / ``query_target_not_a_category`` are defensive guards with no gold fixture (tested by
construction); they never fire on the gold corpus. construction); they never fire on the gold corpus.
""" """