docs(ADR-0164,0165): incremental comprehension reader + regex scope rule (#317 )

Replace the regex sentence-template front-end of the math admissibility
layer with an incremental compositional reader. Lock the architectural
boundary that regex is permitted only at the lexeme level, never as
sentence-structure templates.

ADR-0164 (Proposed) — Incremental Comprehension Reader. Word-by-word
state accumulation over a closed set of semantic categories, with the
operational lexicon living as a pack-shaped data artifact under
language_packs/data/en_core_math_v1/. Reader output type matches the
existing regex parser's output, so the binding-graph admissibility
(ADR-0132/0133/0134/0135), the solver (ADR-0116), and the verifier
(ADR-0117) stay unchanged. wrong=0 is preserved by construction —
the reader produces inputs to the existing admissibility gate, not a
bypass around it. Phased coexistence with the regex layer during
transition; regex sentence templates removed in Phase 3.

ADR-0165 (Proposed) — Regex Scope Rule. Structural invariant: regex
matches one piece of orthographic material with a closed rule
(currency literal, fraction literal, percentage, time-amount, closed
unit-noun sets), never a sentence shape. Lexeme-primitive registry is
closed and grown through the same contemplation -> proposal -> HITL
review corridor that grows vocabulary (ADR-0150 / 0152 / 0155 / 0161).
The engine acquires new recognition tools through reviewed teaching,
not through operator edits to parser code.

ADR-0163's diagnosis (front-end is the bottleneck) is reaffirmed.
Its Phase B-E prescription (regex DerivedRecognizers via
recognizer_match.py) is partially superseded by ADR-0164. ADR-0136
and its S-family (S.1 / S.2 / S.3 / S.4) have the same disposition:
regex sentence-template prescription superseded; empirical refusal
taxonomies and closed-set vocabulary preserved as lexicon seed.
The HITL corridor architecture is preserved; what flows through it
changes from regex recognizers to lexicon entries, categories, and
lexeme primitives.

Session log SESSION-2026-05-26-comprehension-reader.md captures the
narrative of how this decision emerged from the post-D.2 train-sample
baseline review (correct=3 refused=47 wrong=0, 34/47 refusals at the
question gate).

No runtime code changes. ADRs only.

2026-05-26 19:23:05 -07:00

4.3 KiB

Raw Blame History

ADR-0136.S.2 — Conditional-Op Question (Statement-Layer Corridor)

Status: Active — regex patterns scheduled for removal under ADR-0164 Phase 3; closed-set vocabulary preserved as lexicon seed Date: 2026-05-23 Parent: ADR-0136 — see ADR-0136 §Amendment 2026-05-26

Context

After S.0 (context-sentence classifier) and S.1 (rate/event statement parsing) landed, the GSM8K train-sample probe sat at 2/50 admitted (gsm8k-0014, gsm8k-0018), wrong == 0.

A taxonomy pass over the remaining 48 refused cases identified gsm8k-0042 as the next single-barrier unlock:

Ella has 4 bags with 20 apples in each bag and six bags with 25 apples in
each bag.  If Ella sells 200 apples, how many apples does Ella has left?

The initial-state sentence already parses via _CONJ_EMBEDDED_RE (ADR-0131.G.4 embedded quantifier) to InitialPossession(entity="Ella", value=230, unit="apples"). The only barrier is the question form, which neither _Q_ENTITY_RE nor _Q_TOTAL_RE cover: the If <Entity> <verb> <N> <unit>, how many <unit2> does <Entity2> <aux> [left|...]? shape combines a conditional-action operand with the entity-recall question.

Decision

Add a conditional-op question extractor and a corresponding short-circuit in parse_and_solve that:

Matches the closed shape If <Entity> <verb> <N> <unit>, how many <unit2> does <Entity2> <aux> [<qualifier>]?
Classifies <verb> against two closed sets:
- _COND_SUBTRACT_VERBS (sell/sells/sold, give/gives/gave, eat/eats/ate, use, lose, spend, donate, remove, take, send, pay, drop, throw)
- _COND_ADD_VERBS (buy/buys/bought, get/gets/got, receive, find, add, collect, pick, earn, gain)
Refuses on any of: unknown verb, unit mismatch (<unit> vs <unit2> after canonicalization), entity mismatch (<Entity> vs <Entity2> case-insensitively), N <= 0.
In parse_and_solve: if the question yields exactly one CandidateConditionalOpQuestion, collect all extract_initial_candidates from every statement sentence and look for exactly one matching IC by (entity, unit). If found, compute initial_value ± operand by verb polarity; emit only when answer >= 0. Refuses otherwise.

The short-circuit is structurally identical to the S.1 capacity/earnings paths: it bypasses graph construction, returns selected_graph=None, and preserves wrong == 0 by refusing rather than guessing on any ambiguity.

Invariants

admitted_wrong == 0 preserved by:
- Single-match (entity, unit) requirement before emission
- Non-negative answer gate
- Closed verb sets — no wildcards
Context-filler safety rail unchanged (S.0)
No solver/graph/verifier changes — extractor lives in math_candidate_parser.py; short-circuit lives in math_candidate_graph.py

Honest GSM8K delta

Stage	Admitted	Wrong
Pre-S.0	0/50	0
Post-S.1	1/50 (`0014`)	0
Post-S.0 classifier	2/50 (`+0018`)	0
Post-S.2	3/50 (`+0042`)	0

gsm8k-0042 admits with answer == 30.0 (expected: 30).

Consequences

The S.x corridor now spans parser (S.1 capacity/earnings, S.2 question shape) and pre-pass classifier (S.0). Future phases (S.3 compound statements, S.4 coreference) extend this pattern.
The canonical GSM8K runner (evals/gsm8k_math/runner.py) still asserts selected_graph is not None on admission and therefore cannot score the short-circuit admissions. The report.json artifact remains stale (0/50/0); the honest count is asserted via direct parse_and_solve test (test_gsm8k_post_s2_admission_honest). Aligning the canonical runner with the short-circuit paths is deferred (out of S.2 scope).

Deferred

Conditional-op question forms with more than one operand (e.g. "If she gives away 3 and buys 5, …") — needs two-op composition.
Question forms without does <Entity> aux (e.g. "how many apples remain?") — needs a sibling regex.
Conditional questions where the conditional and the question reference different units that are unit-related (e.g. dollars↔cents) — needs unit-relation taxonomy.

4.3 KiB Raw Blame History