core/docs/adr/ADR-0164.4-phase2-statement-frame-reader.md

# ADR-0164.4 — Phase 2 Statement-Frame Reader

**Status:** Proposed
**Date:** 2026-05-26
**Author:** Shay
**Anchor:** [[thesis-decoding-not-generating]]
**Parent:** [ADR-0164 — Incremental Comprehension Reader](./ADR-0164-incremental-comprehension-reader.md)
**Builds on:** [ADR-0164.3 — Cross-Sentence Reading State](./ADR-0164.3-cross-sentence-state.md)
**Related downstream types:** [ADR-0115 — `MathProblemGraph`](./ADR-0115-math-problem-parser-and-graph.md)

---

## Context

Phase 1 (ADR-0164.3) shipped the `question_frame` reader and the
two-level `ProblemReadingState` / `SentenceReadingState` lifecycle. Phase
2 extends the reader to the three statement-side frames and adds the
`finalize()` projection from `ProblemReadingState` into
`MathProblemGraph`, so a whole problem can be read end-to-end without
the legacy regex parser.

## Decision

### Frames ratified

- **`initial_state_frame`** — `entity possession_verb [count] [unit]`,
  emits a `PartialInitialPossession` → `InitialPossession` at
  `finalize()`.
- **`operation_frame`** — `entity (accumulation|depletion|transfer|
  capacity)_verb [count] [unit] [to entity₂]`, emits a `PartialOperation`
  → `Operation` at `finalize()`. `accumulation` → `add`, `depletion` →
  `subtract`, `capacity` → `add`, `transfer` → `transfer`.
- **`descriptive_frame`** — opens on `copula_verb` (or subject-dropped
  verb position), drains known tokens, emits no math state. Used for
  descriptive prose ("Sandra is a baker", "There are some kids in
  camp") that does not bind quantities to operations.

### `finalize()` projection

Operates on a closed `ProblemReadingState`:

1. Require `unknown_target_slot` — else `no_question_target`.
2. Build `entities` tuple from `entity_registry` — empty registry yields
   `dangling_entity`.
3. `PartialInitialPossession` → `InitialPossession` (requires
   `entity`, `quantity.value`, and resolved `unit`).
4. `PartialOperation` → `Operation` (requires `actor`, op-kind from
   verb category, `operand.value`, resolved `unit`).
5. `QuestionTargetSlot` → `Unknown` with `unit` derived from the slot's
   captured unit lemma or the unit-class default.

### Integration flag

`generate.math_candidate_graph.parse_and_solve(text, *,
comprehension_reader: bool = False)` gates the reader-first path. With
the flag `False` (default), behaviour is byte-identical to the existing
regex parser. With the flag `True`, the reader is attempted first; if
any sentence refuses (all-or-nothing) the regex path runs unchanged.

### Wrong = 0 discipline

The reader never returns a graph with a wrong answer in the Phase 2
GSM8K-train sample: any structural ambiguity (multi-quantity ops,
fractions, multi-subject sentences) yields a typed `ReaderRefusal` so
the regex parser handles the case. This preserves the project-wide
`wrong == 0` invariant.

### Lexicon additions ratified (`phase_2_reader_gsm8k_2026-05-26`)

- `currency_unit_noun` +2 (`dollar`, `cent`).
- `accumulation_verb` +2 (`adopt`, `invest`).
- `capacity_verb` +6 (`fill`, `lift`, `play`, `work`, `finish`,
  `drive`).
- `proper_noun_entity_female` +5 (`allison`, `brooke`, `jan`,
  `marion`, `sidney`).
- `proper_noun_entity_male` +14 (`bart`, `fernando`, `georgie`, `jake`,
  `jed`, `jeremie`, `jose`, `orlando`, `rex`, `rudolph`, `steve`,
  `troy`, `xavier`, `yun`).
- `time_unit_noun` +3 (`year`, `month`, `second`).
- `count_unit_noun` +14 (puppy, kitten, parakeet, coconut, macaroon,
  brownie, scoop, section, foot, cable, eraser, crayon, paperclip,
  card, …).
- `drain_token` substantial expansion to absorb prose connectives,
  written numerals, place names, and non-math verbs that should not
  drive frame selection.

### Possessive handling

`_classify()` now strips trailing `'s` and re-attempts lexicon lookup,
so `Rudolph's` resolves to `rudolph` (proper_noun_entity_male). Genitive
possessives drain (e.g., "Aaron and his brother") rather than triggering
the multi-subject refusal.

## Evidence

`evals/gsm8k_math/train_sample/v1/reader_phase2_delta.json` captures
per-case attribution on the 50-case sample:

```text
flag-OFF:  correct=3   wrong=0   refused=47
flag-ON:   correct=3   wrong=0   refused=47
reader_accepted (built a Graph): 3 / 50
```

### Observed bottlenecks

| count | reader refusal class           | examples / interpretation                    |
| ----- | ------------------------------ | -------------------------------------------- |
|   18  | `incomplete_operation` at end  | multi-quantity ops ("4 bags with 20 apples in each bag"); no-quantity op_frame |
|   11  | `unknown_word`                 | `hundred`, `presently`, compound `one-hour`; non-math verbs (`encountered`, `studied`, `holds`) |
|    6  | `unexpected_category`          | fraction/percentage literals (Phase 2.1); multi-subject ("Aaron and Carson") |
|    6  | `unresolved_pronoun`           | `them`, `their`, `his` with no compatible registry entry |
|    5  | `unattached_quantity` at end   | quantity never bound to a unit noun          |
|    1  | `no_question_target` at finalize | question sentence parsed but never set the slot |

### Acceptance gate

The brief asks for `correct ≥ 25 (preferred)` or
`mixed ∈ [4, 24] with documented bottlenecks`. The current count of
`correct = 3` falls below the gate floor. **The reader is structurally
correct (wrong = 0 holds, flag-OFF byte-identical, determinism holds),
but the lexicon and structural coverage are not yet sufficient to clear
the gate.** Closing the gap requires:

1. Phase 2.1: embedded-quantifier aggregates ("N X with M Y in each")
   and multi-quantity operation handling.
2. Phase 2.2: fraction/percentage literal handling (currently refused
   wholesale).
3. Phase 3: multi-subject sentences, descriptive-clause unpacking.

## Invariants

- `versor_condition(F) < 1e-6` — Phase 2 reader does not touch field
  state; trivially preserved.
- `wrong == 0` — empirically verified on the GSM8K train sample under
  both flag settings.
- Flag-OFF byte-identical — `parse_and_solve` short-circuits before any
  reader call when the flag is `False`.
- Determinism — identical input yields identical `trace_hash` on
  repeated runs (50/50 verified).

## Out of scope (refused cleanly in Phase 2)

- Embedded-quantifier aggregates → `embedded-quantifier aggregate;
  deferred to Phase 2.1`.
- Fraction / percentage literals → `unexpected_category` with
  Phase 2.1 deferred-scope detail.
- Conditional frames ("If X, then Y") → Phase 3+.
- Multi-quantity initial state ("Two puppies, two kittens, …") →
  Phase 2.1.
- Multi-subject sentences ("Aaron and Carson saved $40") → Phase 2.1.