docs(ADR-0164,0165): incremental comprehension reader + regex scope rule (#317 )

Replace the regex sentence-template front-end of the math admissibility
layer with an incremental compositional reader. Lock the architectural
boundary that regex is permitted only at the lexeme level, never as
sentence-structure templates.

ADR-0164 (Proposed) — Incremental Comprehension Reader. Word-by-word
state accumulation over a closed set of semantic categories, with the
operational lexicon living as a pack-shaped data artifact under
language_packs/data/en_core_math_v1/. Reader output type matches the
existing regex parser's output, so the binding-graph admissibility
(ADR-0132/0133/0134/0135), the solver (ADR-0116), and the verifier
(ADR-0117) stay unchanged. wrong=0 is preserved by construction —
the reader produces inputs to the existing admissibility gate, not a
bypass around it. Phased coexistence with the regex layer during
transition; regex sentence templates removed in Phase 3.

ADR-0165 (Proposed) — Regex Scope Rule. Structural invariant: regex
matches one piece of orthographic material with a closed rule
(currency literal, fraction literal, percentage, time-amount, closed
unit-noun sets), never a sentence shape. Lexeme-primitive registry is
closed and grown through the same contemplation -> proposal -> HITL
review corridor that grows vocabulary (ADR-0150 / 0152 / 0155 / 0161).
The engine acquires new recognition tools through reviewed teaching,
not through operator edits to parser code.

ADR-0163's diagnosis (front-end is the bottleneck) is reaffirmed.
Its Phase B-E prescription (regex DerivedRecognizers via
recognizer_match.py) is partially superseded by ADR-0164. ADR-0136
and its S-family (S.1 / S.2 / S.3 / S.4) have the same disposition:
regex sentence-template prescription superseded; empirical refusal
taxonomies and closed-set vocabulary preserved as lexicon seed.
The HITL corridor architecture is preserved; what flows through it
changes from regex recognizers to lexicon entries, categories, and
lexeme primitives.

Session log SESSION-2026-05-26-comprehension-reader.md captures the
narrative of how this decision emerged from the post-D.2 train-sample
baseline review (correct=3 refused=47 wrong=0, 34/47 refusals at the
question gate).

No runtime code changes. ADRs only.

2026-05-26 19:23:05 -07:00

3.2 KiB

Raw Permalink Blame History

ADR-0136.S.1 — Rate/Event Statement Parsing

Status: Accepted — regex patterns scheduled for removal under ADR-0164 Phase 3; closed-set vocabulary preserved as lexicon seed Parent: ADR-0136 (Statement Layer Corridor) — see ADR-0136 §Amendment 2026-05-26 Date: 2026-05-23

Context

The GSM8K refusal taxonomy (evals/gsm8k_math/train_sample/v1/refusal_taxonomy.json) reveals that 23/50 cases are blocked by context-filler sentences (correctly refused — no parseable numeric state), while 4/50 have rate/capacity/price as their primary barrier. The remaining cases are compound-statement, distributive-multiply, and diverse long-tail shapes.

This ADR targets the 4 rate-class cases with two closed statement shapes.

Taxonomy Finding

Primary barrier	Cases	S.1 scope?
`context_filler`	23	No — correctly refused
rate/capacity/price	4	Yes
`compound_statement`	5	No
`distributive_multiply`	1 (+5 secondary)	No
diverse long-tail	17	No

Closed Verb Sets

Capacity verbs: shuck, pick, pack, make, produce, type, read, write, paint, run, score, answer, complete (+ third-person -s forms).

Earnings verbs: make, earn, receive, get, charge (+ third-person -s forms).

No regex wildcards for verbs — every admitted verb is explicitly listed in a frozen set. Sentences with verbs outside the closed set are refused (not wrong).

Short-Circuit Rationale

Both rate shapes bypass the Cartesian-product candidate graph because the rate computation is a direct rate × time multiplication with unit conversion, not a graph of initial-possessions and operations. The short-circuit runs before _filtered_statement_choices so that rate-shaped sentences don't trigger the "no admissible candidate" refusal.

Actor matching is required: capacity questions with pronouns (he/she) accept any actor; named-actor questions require case-insensitive match. Mismatched actors produce refusal, not wrong answers.

Honest GSM8K Claim

Pre-S.1: 0/50 admitted (all refused).
Post-S.1: 1/50 admitted — gsm8k-0014 (Bob shucks oysters) with answer 240.0 (correct).
admitted_wrong = 0 (safety rail preserved).

The other 3 rate-class cases remain blocked by context-filler sentences in their opening statements; the rate parsing behind them is irrelevant until those sentences parse.

Deferred

Context-filler gated problems (23 cases — needs semantic classification of narrative scene-setter sentences).
Conditional branching (overtime rules, e.g. "if she works more than 8 hours").
Percentage/interest rates (10% simple interest).
Multi-statement earnings (duration asserted in a separate sentence from the rate — needs general duration-statement parser).

Evidence

Axis lane: evals/math_capability_axes/S1_rate_events/v1/ — 20/20 pass, wrong=0.
B3 bounded-grammar lane: unchanged (wrong=0).
GSM8K candidate-graph probe: wrong=0, admitted=1/50.
Tests: tests/test_adr_0136_S1_rate_events.py — ≥15 tests including B3 regression guard and GSM8K admitted_wrong=0 rail.

3.2 KiB Raw Permalink Blame History Unescape Escape