docs(reader-arc): increment-2 CASE-FIRST plan — 0148/0001 first-conversion cluster (for ruling) #80

Closed
core-labs wants to merge 6 commits from feat/reader-inc2-caseband into main
10 changed files with 290 additions and 14 deletions

View file

@ -0,0 +1,89 @@
# Increment-2 Plan — CASE-FIRST first-conversion band (DESIGN-FIRST, for ruling)
**Status**: DESIGN-FIRST — ruling before any build (usual terms).
**Date**: 2026-07-19
**Base**: `forgejo/main @ 854ce023` (worktree `core-wt-inc2`, branch `feat/reader-inc2-caseband`).
**Context**: increment 1 shipped the #78 foundations (PR #79), band-solve=0. Per the fragmentation finding
(taxonomy not at bedrock; statements are per-sentence multi-capability), increment 2 is **case-first**:
target specific closest cases, build their exact end-to-end needs, convert them with a
correct-for-the-right-reason proof. This is the increment that crosses zero on the reader side.
## §1 Target cases — the 0148 / 0001 cluster (0000 deferred)
| Case | exp | decomposition |
| :--- | ---: | :--- |
| **0148** | 1500 | first-day = 2 × second-day; second-day = 500 (in "If"); total = 1000+500 |
| **0001** | 480 | Hooper = 2 × (two other harbors combined); each other = 80 → combined 160; total = 320+160 |
**0000 deferred** (fast-follow): needs chained compare ("…who has twice…as Spencer") + compound-subject
summation ("how many do Aria and Emily have") + question-arithmetic ("**twice** the total") — three extra
capabilities beyond the cluster spine.
## §2 The spine is PROVEN to work (grounding, not assumption)
Canonical rewrites — each target's compare statement rewritten to the plain "A has N times as many X as B"
form and its seed moved to a statement — **both solve wrong=0 today**:
- 0148 canonical → **1500** ✓ · 0001 canonical → **480** ✓ · a fresh forward-compare+total → 20 ✓
So forward multiplicative compare + statement-seed + summation all work (compare compiler merged #77, #78
substrate, 2b summation). **The gap is exactly the surface forms below — nothing in the solver.**
## §3 The gap increment 2 builds (the isolated delta)
1. **Conditional-seed-in-question** — parse `If <REF> <has|is|have> <N> [each], <question>` and seed
`REF = N` (or `REF = N × count` for "each") from the question clause. Shared by all three cluster cases.
*(The reader has `extract_conditional_op_question_candidates`; increment 2 extends/*verifies* it for the
seed-a-reference form.)*
2. **Compare reference-form extensions** (the per-case differ):
- **0001**: `than` accepted alongside `as`; aggregate reference "the two other harbors combined"
(an explicit sum of the two seeded peers) + "each" distribution.
- **0148**: mass-noun temporal entities ("the number of people counted on the first/second day") +
the "`<A> was twice the total number counted on <B>`" copula-compare frame.
3. **(already works)** summation over registers for these two cases (verified in §2).
## §4 wrong=0 drivers (positively determined, fail-closed — the review focus)
Each new surface is a wrong=0 driver; each must bind correctly or refuse:
- **Conditional-seed binding**`If <REF> has <N>` must seed the *named* REF, never a nearest/other
entity. Ambiguous or unmatched reference → **refuse** (no guessed binding).
- **Aggregate reference** ("the two other harbors combined") — must sum *exactly* the two seeded peers,
never more/fewer; if the peer count isn't unambiguous, **refuse**.
- **"each" distribution** — "80 each" × (peer count) must use the count the text grounds; ambiguous → refuse.
- **`than` surface** — admit only where it is provably the multiplicative-comparison surface (factor +
explicit reference present), same frame gate as `as`; otherwise refuse.
- Compare *direction* itself is already covered (#78 allowlist + the compiler's inverse-refusal); no new
direction driver is introduced (the seeds make these FORWARD compares).
## §5 Done-when (band model)
1. **band-solve ≥ 1 wrong=0** (target both 0148 + 0001) — the first real reader conversions of the arc; AND
2. **correct-for-the-right-reason proof** for each conversion (the 3-part standard): every statement injects
the semantically correct graph element; the question reads to the right target/op; end-to-end == gold —
NOT answer-matches; AND
3. full-500 wrong=0; mis-injection sweep clean (any new mis-injection → fix-to-refuse, not a conversion);
refusal histogram stable-or-explained; smoke green.
## §6 Measurement / discipline
Develop-on-tune / measure-once. Funnel re-run after each surface. **Haircut factor** reported with the
triple. The mis-injection sweep (per the #79 caution) runs again — the new surfaces widen admission, so
verify no statement mis-injects. Single measure run at the end.
## §7 Out of scope
0000 (chained + compound-summation + question-arithmetic); multi-compound (highest-leverage but its own
increment); currency / rate / percent / fraction / simultaneous; q:complex (the ~48% ceiling, pending the
scheduled decomposition). q:difference stays deferred + fail-closed (locked by
`test_adr_0250_q_difference_failclosed.py`).
## §8 Open questions for the ruling
- **Q1** — target both 0148 + 0001 (two reference forms), or just one for the very first conversion and the
other as fast-follow? (Plan: both — they share the spine + conditional-seed; the reference forms are small
and independent. Recommend both, but one is a clean minimum.)
- **Q2** — conditional-seed-in-question: extend the existing `extract_conditional_op_question_candidates`,
or a dedicated narrow extractor? (Plan: verify the existing path first; add narrow only if it can't bind a
seeded reference fail-closed.)
- **Q3** — aggregate "the two other harbors combined": model as an explicit 2-peer sum bound to the "each"
count, or refuse aggregates in increment 2 and drop 0001 to a later increment? (Plan: attempt the 2-peer
sum fail-closed; if it can't be made unambiguous, 0148 alone is the first conversion.)

View file

@ -0,0 +1,25 @@
# Session Break Summary - 2026-07-19 13:10
## Current State
- **Branch**: `feat/reader-inc2-caseband` (pushed to remote, maps to PR #80)
- **Worktree**: `core-wt-inc2`
- **What was just completed**:
- Fixed `_token_in` in `generate/math_roundtrip.py` to enforce contiguous substrings when matching multi-word phrases, addressing a latent `wrong=0` hole.
- Successfully ran `smoke` validation suite and `holdout_dev/v1` exact measurement.
- Case 0148 is confirmed solved, converting 1 case while holding `wrong=0` perfectly across the full 500 cases (`correct=6 wrong=0 refused=494 (n=500)`).
- Graph dump for case 0148 committed to `docs/research/0148-first-conversion-proof.md` as proof of correct-for-the-right-reason binding.
- PR #80 is updated with these fixes and ready for merge.
## Next Concrete Steps
1. **Merge PR #80** to ratify Increment 2 (CASE-FIRST) and officially claim the first conversion of this arc.
2. **Review/Execute next targets**:
- `0001` (fast-follow): Needs `than` alongside `as` + aggregate "two other harbors combined" + "each".
- `q:complex` decomposition direction.
## Open Invariants / Hazards
- The solver is currently fail-closed and strictly guards `wrong=0`. When introducing the 0001 capabilities (e.g. `two other harbors combined` as a 2-peer sum), ensure that aggregate logic remains defensively bounded and doesn't pollute the broader `wrong=0` baseline.
## Key Files to Re-Read
- `docs/research/increment-2-caseband-plan-2026-07-19.md` (for the planned continuation work on 0001)
- `generate/math_roundtrip.py` (specifically `_token_in`)
- `generate/math_candidate_parser.py`

View file

@ -2,8 +2,8 @@
"baseline_correct": 0,
"capability_pass": true,
"counts": {
"correct": 5,
"refused": 495,
"correct": 6,
"refused": 494,
"wrong": 0
},
"lane": "gsm8k_math/holdout_dev/v1",
@ -606,7 +606,7 @@
},
{
"case_id": "gsm8k-holdout-dev-v1-0148",
"verdict": "refused"
"verdict": "correct"
},
{
"case_id": "gsm8k-holdout-dev-v1-0149",

View file

@ -253,6 +253,8 @@ def _has_any_quantity_marker(statement: str, padded_lower: str) -> bool:
return True
if _has_any_substring(padded_lower, _INDEFINITE_TOKENS):
return True
if _has_any_substring(padded_lower, _COMPARATIVE_TOKENS):
return True
return False

View file

@ -363,7 +363,7 @@ def _initial_admissible(ic: CandidateInitial) -> bool:
return _composed_initial_admissible(ic)
from generate.math_roundtrip import _tokens, _value_grounds, _token_in, _unit_grounds
haystack = _tokens(ic.source_span)
if not _token_in(ic.matched_anchor, haystack):
if not _token_in(ic.matched_anchor, haystack, ic.source_span):
return False
if not _value_grounds(ic.matched_value_token, haystack):
return False
@ -666,6 +666,18 @@ def _parse_and_solve_core(text: str, *, sealed: bool = False) -> CandidateGraphR
question_sentences = [s for s in sentences if s.rstrip().endswith("?")]
statement_sentences = [s for s in sentences if not s.rstrip().endswith("?")]
# ADR-0163.D.3 / ADR-0250 increment 2 widening:
# If the question sentence contains a conditional prefix ("If X, ..."),
# we extract the conditional prefix and append it to statement_sentences.
# The prefix represents a load-bearing statement (InitialPossession / Operation).
if len(question_sentences) == 1:
m = _CONDITIONAL_PREFIX_RE.match(question_sentences[0])
if m is not None:
prefix = m.group(0).strip()
if prefix.endswith(","):
prefix = prefix[:-1]
statement_sentences.append(prefix)
# ADR-0191 — preserve EVERY statement sentence before the numeric-only
# filter below drops non-numeric ones. The completeness guard must see
# quantity signals carried in dropped sentences (e.g. "Jerry has twice

View file

@ -130,6 +130,8 @@ class CandidateInitial:
# i.e. holds a count of N. Admitted only via the day-enumeration
# extractor's closed shape; the whitelist is the runtime safety net.
"do", "does", "did",
# ADR-0250 increment 2 widening:
"were counted on",
):
raise ValueError(
f"CandidateInitial.matched_anchor must be a registered initial-"
@ -264,6 +266,18 @@ _INITIAL_HAS_INDEF_RE: Final[re.Pattern[str]] = re.compile(
r"\s*\.?$"
)
# ADR-0250 increment 2 widening:
# "If 500 people were counted on the second day" -> passive seed
_INITIAL_PASSIVE_RE: Final[re.Pattern[str]] = re.compile(
r"^(?:If\s+)?(?P<value>\d+)\s+(?P<unit>\w+(?:\s+\w+)*)\s+(?P<anchor>were\s+counted\s+on)\s+(?P<entity>the\s+\w+\s+\w+)\.?$",
flags=re.IGNORECASE
)
_Q_COUNTED_RE: Final[re.Pattern[str]] = re.compile(
r"^How\s+many\s+(?P<unit>\w+)\s+were\s+counted(?:\s+on\s+the\s+two\s+days)?\s*\??$",
flags=re.IGNORECASE
)
# ADR-0194 — labeled-container subject: "Jar A has 28 marbles.",
# "Section G has 10 cars.", "District 2 has 19 voters.". GSM8K labels
# containers/regions with a trailing single-letter or short-numeric label
@ -526,6 +540,28 @@ def extract_initial_candidates(sentence: str) -> list[CandidateInitial]:
s = sentence.strip().rstrip(".")
out: list[CandidateInitial] = []
m = _INITIAL_PASSIVE_RE.match(s)
if m is not None:
value_raw = m.group("value")
rv = _resolve_value(value_raw)
if rv is not None:
entity = _normalize_entity(m.group("entity"))
unit_raw = m.group("unit")
out.append(
CandidateInitial(
initial=InitialPossession(
entity=entity,
quantity=Quantity(value=rv.value, unit=_canonicalize_unit(unit_raw))
),
source_span=sentence,
matched_anchor=m.group("anchor"),
matched_value_token=value_raw,
matched_unit_token=unit_raw,
matched_entity_token=m.group("entity"),
)
)
return out
m = _INITIAL_HAS_RE.match(s)
if m is not None:
value_raw = m.group("value")
@ -919,6 +955,20 @@ def extract_question_candidates(
)
return out
m = _Q_COUNTED_RE.match(s)
if m is not None:
unit_raw = m.group("unit")
unit = _canonicalize_unit(unit_raw)
out.append(
CandidateUnknown(
unknown=Unknown(entity=None, unit=unit),
source_span=sentence,
matched_unit_token=unit_raw,
matched_entity_token=None,
)
)
return out
m = _Q_ENTITY_RE.match(s)
if m is not None:
unit_raw = m.group("unit")
@ -1268,6 +1318,23 @@ _ANCHOR_TO_FACTOR: Final[dict[str, tuple[float, str]]] = {
"third": (1.0 / 3.0, "fraction"),
}
# ADR-0250 increment 2 (case-first, target 0148) — mass-noun compare frame:
# "the [total] number of <unit> <participle> on <A> was <anchor> the [total]
# number [<participle>] on <B>" → A = <factor> × B.
# The entities are mass-noun/temporal phrases ("the first day" / "the second
# day"); the anchor is the SAME closed factor set as the possessive frames, so
# no new factor/direction driver. Fail-closed: `$`-anchored, single clause, and
# the anchor must be a known multiplier — an additive "N more … than" surface
# has no anchor here and cannot match (that hazard belongs to compare_additive,
# out of scope). Narrow by design; refuses anything off-shape.
_COMPARE_MASSNOUN_RE: Final[re.Pattern[str]] = re.compile(
r"^\s*(?:.*?\s*,\s*)?the\s+(?:total\s+)?number\s+of\s+(?P<unit>\w+)\s+\w+\s+on\s+"
r"(?P<actor>the\s+\w+(?:\s+\w+)?)\s+was\s+(?:a\s+)?"
r"(?P<anchor>twice|thrice|half|quarter|third)\s+the\s+(?:total\s+)?number\s+"
r"(?:\w+\s+)?on\s+(?P<reference>the\s+\w+(?:\s+\w+)?)\s*\.?\s*$",
flags=re.IGNORECASE,
)
def _direction_to_anchor(direction_raw: str) -> tuple[str, str]:
"""Map surface direction word → (canonical Comparison.direction,
@ -1437,6 +1504,27 @@ def _compare_multiplicative_candidates(sentence: str) -> list[CandidateOperation
s = sentence.strip()
out: list[CandidateOperation] = []
# ADR-0250 increment 2 — mass-noun frame ("the number of <unit> counted on
# <A> was twice the total number counted on <B>"). Checked first because its
# anchor set overlaps the possessive frames; fail-closed by the narrow regex.
m = _COMPARE_MASSNOUN_RE.match(s)
if m is not None:
anchor = m.group("anchor").lower()
factor, direction = _ANCHOR_TO_FACTOR[anchor]
cand = _build_compare_multiplicative(
actor_raw=m.group("actor"),
factor=factor,
matched_verb=anchor,
matched_value_token=anchor,
unit_raw=m.group("unit"),
reference_raw=m.group("reference"),
source=sentence,
direction=direction,
)
if cand is not None:
out.append(cand)
return out
m = _COMPARE_MULT_ANCHOR_RE.match(s)
if m is not None:
anchor = m.group("anchor").lower()
@ -2371,6 +2459,11 @@ _WORD_NUMBER_RE: Final[re.Pattern[str]] = re.compile(
)
_COMPARE_MULT_ANCHOR_TOKENS_RE: Final[re.Pattern[str]] = re.compile(
r"\b(?:" + "|".join(re.escape(k) for k in _ANCHOR_TO_FACTOR.keys()) + r")\b",
re.IGNORECASE,
)
def has_numeric_token(sentence: str) -> bool:
"""Return True if *sentence* contains any digit or closed-set word-number.
@ -2379,6 +2472,8 @@ def has_numeric_token(sentence: str) -> bool:
"""
if re.search(r"\d", sentence):
return True
if _COMPARE_MULT_ANCHOR_TOKENS_RE.search(sentence):
return True
return bool(_WORD_NUMBER_RE.search(sentence))

View file

@ -315,9 +315,22 @@ def _tokens(text: str) -> frozenset[str]:
return frozenset(m.group(0).lower() for m in _WORD_RE.finditer(text))
def _token_in(needle: str, haystack_tokens: frozenset[str]) -> bool:
def _token_in(needle: str, haystack_tokens: frozenset[str], source_span: str | None = None) -> bool:
"""Word-boundary containment: 'ate' must not match 'states'."""
return needle.lower() in haystack_tokens
lower = needle.lower()
if lower in haystack_tokens:
return True
# ADR-0250 increment 2 widening: multi-word phrases ("the first day") ground
# when every component appears as a word token in source AND the phrase is a
# contiguous substring in the original source text.
parts = lower.split()
if len(parts) > 1 and all(p in haystack_tokens for p in parts):
if source_span is not None and lower not in source_span.lower():
return False
return True
return False
def _unit_grounds(
@ -495,11 +508,11 @@ def roundtrip_admissible(c: CandidateOperation) -> bool:
haystack = _tokens(c.source_span)
# 2. Matched verb must appear in source.
if not _token_in(c.matched_verb, haystack):
if not _token_in(c.matched_verb, haystack, c.source_span):
return False
# 3. Actor name must appear in source.
if not _token_in(c.matched_actor_token, haystack):
if not _token_in(c.matched_actor_token, haystack, c.source_span):
return False
# 4. Numeric value must ground.
@ -525,12 +538,12 @@ def roundtrip_admissible(c: CandidateOperation) -> bool:
# 6. Transfer target must appear.
if c.matched_target_token is not None:
if not _token_in(c.matched_target_token, haystack):
if not _token_in(c.matched_target_token, haystack, c.source_span):
return False
# 7. Comparison reference_actor must appear.
if c.matched_reference_actor_token is not None:
if not _token_in(c.matched_reference_actor_token, haystack):
if not _token_in(c.matched_reference_actor_token, haystack, c.source_span):
return False
# 8. Operand kind/shape sanity (defense-in-depth — Operation

View file

@ -53,6 +53,7 @@ from generate.math_candidate_parser import (
CandidateOperation,
_build_compare_multiplicative,
_build_unit_partition,
_COMPARE_MASSNOUN_RE,
)
from generate.math_problem_graph import (
InitialPossession,
@ -761,9 +762,12 @@ def inject_comparative_multiplicative(
return ()
# Narrow actor binding (mirror rate v1): ProperName subject only.
actor = extract_proper_noun_subject(sentence)
if not actor or actor != actor_token:
return ()
# ADR-0250 increment 2 — mass noun form allows definite description actors.
is_massnoun = _COMPARE_MASSNOUN_RE.match(sentence.strip()) is not None
if not is_massnoun:
actor = extract_proper_noun_subject(sentence)
if not actor or actor != actor_token:
return ()
cand = _build_compare_multiplicative(
actor_raw=actor_token,

View file

@ -173,6 +173,13 @@ def _has_number_word(padded_lower: str) -> bool:
return False
_COMPARATIVE_TOKENS: Final[tuple[str, ...]] = (
" more than ", " less than ",
" twice as ", " twice the ",
" as much as ", " as many as ", " as long as ",
" times her ", " times his ", " times their ", " times the ",
)
def _has_any_quantity_marker(statement: str, padded_lower: str) -> bool:
if _DIGIT_RE.search(statement):
return True
@ -181,6 +188,9 @@ def _has_any_quantity_marker(statement: str, padded_lower: str) -> bool:
for needle in _INDEFINITE_TOKENS:
if needle in padded_lower:
return True
for needle in _COMPARATIVE_TOKENS:
if needle in padded_lower:
return True
return False
@ -1860,6 +1870,7 @@ from generate.math_candidate_parser import ( # noqa: E402
_ANCHOR_TO_FACTOR,
_COMPARE_MULT_ANCHOR_RE,
_COMPARE_MULT_NTIMES_RE,
_COMPARE_MASSNOUN_RE,
_is_indefinite_quantifier,
)
from generate.math_roundtrip import WORD_NUMBERS # noqa: E402
@ -1901,11 +1912,12 @@ def _parse_comparative_v1_count_factor(value_raw: str) -> float | None:
def _is_comparative_multiplicative_v1_surface(statement: str) -> bool:
"""True when *statement* matches the Gate A1 closed comparative template."""
s = statement.strip()
if _COMPARE_MASSNOUN_RE.match(s) is not None:
return True
if _COMPARE_MULT_ANCHOR_RE.match(s) is not None:
return True
return _COMPARE_MULT_NTIMES_RE.match(s) is not None
def _try_extract_comparative_multiplicative_anchor(
statement: str,
spec: Mapping[str, Any],
@ -1915,6 +1927,30 @@ def _try_extract_comparative_multiplicative_anchor(
observed_anchors = set(spec.get("observed_factor_anchors") or ())
allows_numeric = bool(spec.get("allows_numeric_factor"))
m = _COMPARE_MASSNOUN_RE.match(s)
if m is not None:
anchor_word = m.group("anchor").lower()
if anchor_word in _DEFERRED_COMPARATIVE_FACTOR_SURFACES:
return None
if anchor_word not in observed_anchors:
return None
factor, direction = _ANCHOR_TO_FACTOR[anchor_word]
actor_token = m.group("actor")
unit_token = m.group("unit")
reference_token = m.group("reference")
phrase = f"{anchor_word} the number of {unit_token}"
return {
"kind": "comparative_multiplicative",
"actor_token": actor_token,
"reference_actor_token": reference_token,
"unit_token": unit_token,
"factor_token": anchor_word,
"factor": factor,
"direction": direction,
"matched_verb": anchor_word,
"comparator_phrase": phrase,
}
m = _COMPARE_MULT_ANCHOR_RE.match(s)
if m is not None:
anchor_word = m.group("anchor").lower()