core/docs/decisions/ADR-0164.4-phase2-statement-frame-reader.md

ADR-0164.4 — Phase 2 Statement-Frame Reader

Status: Proposed Date: 2026-05-26 Author: Shay Anchor: thesis-decoding-not-generating Parent: ADR-0164 — Incremental Comprehension Reader Builds on: ADR-0164.3 — Cross-Sentence Reading State Related downstream types: ADR-0115 — MathProblemGraph

---

Context

Phase 1 (ADR-0164.3) shipped the question_frame reader and the two-level ProblemReadingState / SentenceReadingState lifecycle. Phase 2 extends the reader to the three statement-side frames and adds the finalize() projection from ProblemReadingState into MathProblemGraph, so a whole problem can be read end-to-end without the legacy regex parser.

Decision

Frames ratified

• initial_state_frame — entity possession_verb [count] [unit], emits a PartialInitialPossession → InitialPossession at finalize().
• operation_frame — entity (accumulation|depletion|transfer| capacity)_verb [count] [unit] [to entity₂], emits a PartialOperation → Operation at finalize(). accumulation → add, depletion → subtract, capacity → add, transfer → transfer.
• descriptive_frame — opens on copula_verb (or subject-dropped verb position), drains known tokens, emits no math state. Used for descriptive prose ("Sandra is a baker", "There are some kids in camp") that does not bind quantities to operations.

finalize() projection

Operates on a closed ProblemReadingState:

1. Require unknown_target_slot — else no_question_target.
2. Build entities tuple from entity_registry — empty registry yields dangling_entity.
3. PartialInitialPossession → InitialPossession (requires entity, quantity.value, and resolved unit).
4. PartialOperation → Operation (requires actor, op-kind from verb category, operand.value, resolved unit).
5. QuestionTargetSlot → Unknown with unit derived from the slot's captured unit lemma or the unit-class default.

Integration flag

generate.math_candidate_graph.parse_and_solve(text, *, comprehension_reader: bool = False) gates the reader-first path. With the flag False (default), behaviour is byte-identical to the existing regex parser. With the flag True, the reader is attempted first; if any sentence refuses (all-or-nothing) the regex path runs unchanged.

Wrong = 0 discipline

The reader never returns a graph with a wrong answer in the Phase 2 GSM8K-train sample: any structural ambiguity (multi-quantity ops, fractions, multi-subject sentences) yields a typed ReaderRefusal so the regex parser handles the case. This preserves the project-wide wrong == 0 invariant.

Lexicon additions ratified (phase_2_reader_gsm8k_2026-05-26)

• currency_unit_noun +2 (dollar, cent).
• accumulation_verb +2 (adopt, invest).
• capacity_verb +6 (fill, lift, play, work, finish, drive).
• proper_noun_entity_female +5 (allison, brooke, jan, marion, sidney).
• proper_noun_entity_male +14 (bart, fernando, georgie, jake, jed, jeremie, jose, orlando, rex, rudolph, steve, troy, xavier, yun).
• time_unit_noun +3 (year, month, second).
• count_unit_noun +14 (puppy, kitten, parakeet, coconut, macaroon, brownie, scoop, section, foot, cable, eraser, crayon, paperclip, card, …).
• drain_token substantial expansion to absorb prose connectives, written numerals, place names, and non-math verbs that should not drive frame selection.

Possessive handling

_classify() now strips trailing 's and re-attempts lexicon lookup, so Rudolph's resolves to rudolph (proper_noun_entity_male). Genitive possessives drain (e.g., "Aaron and his brother") rather than triggering the multi-subject refusal.

Evidence

evals/gsm8k_math/train_sample/v1/reader_phase2_delta.json captures per-case attribution on the 50-case sample:

flag-OFF:  correct=3   wrong=0   refused=47
flag-ON:   correct=3   wrong=0   refused=47
reader_accepted (built a Graph): 3 / 50

Observed bottlenecks

count	reader refusal class	examples / interpretation
18	incomplete_operation at end	multi-quantity ops ("4 bags with 20 apples in each bag"); no-quantity op_frame
11	unknown_word	hundred, presently, compound one-hour; non-math verbs (encountered, studied, holds)
6	unexpected_category	fraction/percentage literals (Phase 2.1); multi-subject ("Aaron and Carson")
6	unresolved_pronoun	them, their, his with no compatible registry entry
5	unattached_quantity at end	quantity never bound to a unit noun
1	no_question_target at finalize	question sentence parsed but never set the slot

Acceptance gate

The brief asks for correct ≥ 25 (preferred) or mixed ∈ [4, 24] with documented bottlenecks. The current count of correct = 3 falls below the gate floor. The reader is structurally correct (wrong = 0 holds, flag-OFF byte-identical, determinism holds), but the lexicon and structural coverage are not yet sufficient to clear the gate. Closing the gap requires:

1. Phase 2.1: embedded-quantifier aggregates ("N X with M Y in each") and multi-quantity operation handling.
2. Phase 2.2: fraction/percentage literal handling (currently refused wholesale).
3. Phase 3: multi-subject sentences, descriptive-clause unpacking.

Invariants

• versor_condition(F) < 1e-6 — Phase 2 reader does not touch field state; trivially preserved.
• wrong == 0 — empirically verified on the GSM8K train sample under both flag settings.
• Flag-OFF byte-identical — parse_and_solve short-circuits before any reader call when the flag is False.
• Determinism — identical input yields identical trace_hash on repeated runs (50/50 verified).

Out of scope (refused cleanly in Phase 2)

• Embedded-quantifier aggregates → embedded-quantifier aggregate; deferred to Phase 2.1.
• Fraction / percentage literals → unexpected_category with Phase 2.1 deferred-scope detail.
• Conditional frames ("If X, then Y") → Phase 3+.
• Multi-quantity initial state ("Two puppies, two kittens, …") → Phase 2.1.
• Multi-subject sentences ("Aaron and Carson saved $40") → Phase 2.1.