feat(ADR-0163.D.3): conditional-prefix recovery for question admission (#308)

Phase D made statement-level admission consult the ratified
recognizer registry (PR #302) but the same wiring at the
question-admissibility point was left for follow-up.  Post-Phase-B
round-2 ratification, 38 of 47 still-refused GSM8K train_sample
cases now refuse on QUESTIONS (vs 7 pre-ratification) — the
architectural bottleneck has migrated downstream.

The biggest single still-refused question shape is
``nested_question_target`` (11 of 38 cases): ``If X, how many Y
does Z have?`` style.  The existing ``_Q_ENTITY_RE`` regex only
matches ``How many UNIT does ENTITY have`` without a conditional
prefix.

D.3 adds a deterministic, pure prefix-strip step that runs ONLY
when the bare parser returns no candidates:

  _filtered_question_choices:
    candidates = existing parser
    if empty AND sentence starts with "If X, ":
      strip the prefix, upper-case the first letter
      re-run the existing parser on the suffix

Tests pin: prefix-strip correctness on the 5 brief-mandated case
shapes, no false admissions when the suffix is still unparseable,
non-question pass-through unchanged, idempotency, no input
mutation, real-GSM8K-question parameterised coverage.

Empirical reality (verified by re-running the train_sample lane):
the strip operation succeeds deterministically on every
nested_question_target case, but the resulting suffix still hits
OTHER parser limitations (``how much`` mass nouns instead of
``how many`` units, modal verbs like ``will be able to``, pronoun
entities, additional clause prefixes).  D.3 alone produces ZERO
additional case-level lift on the current parser regex.  D.3 is
necessary-but-not-sufficient; the next layer (extending the
question grammar to mass nouns + non-"have" verbs + pronoun
entity resolution) is required for the conditional-question
cases to compose into correct answers.

That layer is a separate ADR — it touches grammar surface, not
admission wiring.  This PR ships ONLY the wiring extension.

Validation:
- 43 new + existing tests passed: tests/test_adr_0163_d3_*,
  tests/test_math_candidate_graph,
  tests/test_candidate_graph_recognizer_wiring
- 222 capability-axis tests passed / 2 pre-existing main
  failures / 3 skipped — G1..G5 + S1 wrong=0 byte-identical
- 67 smoke passed

wrong=0 invariant preserved by construction: recovered candidates
flow through the same _question_admissible gate as direct
candidates; no new admission paths bypass the structural check.

Scope: extends one function in generate/math_candidate_graph.py.
Does not modify the parser regexes, the solver, or the recognizer
registry.
This commit is contained in:
Shay 2026-05-26 15:40:49 -07:00 committed by GitHub
parent 1f5ffcf6c7
commit b568ab6c3d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 170 additions and 1 deletions

View file

@ -166,14 +166,53 @@ def _filtered_statement_choices(sentence: str) -> list[SentenceChoice]:
def _filtered_question_choices(sentence: str) -> list[CandidateUnknown]:
"""Return all admissible question candidates after the question-
specific structural check."""
specific structural check.
ADR-0163.D.3 conditional-prefix recovery. When the existing
parser returns no candidates AND the question begins with an
"If X, ..." conditional prefix, strip the prefix and re-try.
This admits the ``nested_question_target`` shape that the bare
regex misses (11 of 38 GSM8K train_sample post-Phase-D question
refusals share this shape). Skip-only safety: if the stripped
question still produces no admissible candidate, refuse as before.
"""
out: list[CandidateUnknown] = []
for qc in extract_question_candidates(sentence):
if _question_admissible(qc):
out.append(qc)
if not out:
stripped = _strip_conditional_prefix(sentence)
if stripped is not None and stripped != sentence:
for qc in extract_question_candidates(stripped):
if _question_admissible(qc):
out.append(qc)
return out[:MAX_CANDIDATES_PER_SENTENCE]
_CONDITIONAL_PREFIX_RE: re.Pattern[str] = re.compile(
r"^\s*[Ii]f\s+.+?,\s+(?=[A-Za-z])",
)
def _strip_conditional_prefix(sentence: str) -> str | None:
"""ADR-0163.D.3 — remove an ``If X, `` conditional prefix.
Returns the suffix with its first letter upper-cased when the
pattern matches; returns ``None`` if no conditional prefix is
present. The transformation is deterministic and pure.
"""
m = _CONDITIONAL_PREFIX_RE.match(sentence)
if m is None:
return None
suffix = sentence[m.end():]
if not suffix:
return None
# Existing question regexes expect a leading "How" (case-insensitive
# in pattern); upper-case the first character to mirror the
# canonical surface form so the deterministic match holds.
return suffix[0].upper() + suffix[1:]
def _initial_admissible(ic: CandidateInitial) -> bool:
"""Light structural ground-check for initial-possession candidates.

View file

@ -0,0 +1,130 @@
"""ADR-0163.D.3 — conditional-prefix recovery for question admission.
When a question carries an ``If X, ...`` conditional prefix, the
existing question regex misses it. Phase D.3 strips the prefix and
re-tries via the same admission path. Result: the
``nested_question_target`` shape (11 of 38 GSM8K train_sample
question refusals post-Phase-D) becomes admissible without changing
the underlying parser regex or the solver.
The transformation is deterministic and pure. wrong=0 invariant is
preserved by construction a recovered candidate flows through the
same ``_question_admissible`` gate as any other parser output.
"""
from __future__ import annotations
import pytest
from generate.math_candidate_graph import (
_filtered_question_choices,
_strip_conditional_prefix,
)
# ---------------------------------------------------------------------------
# _strip_conditional_prefix is pure + deterministic
# ---------------------------------------------------------------------------
class TestStripConditionalPrefix:
def test_strips_canonical_if_prefix(self) -> None:
out = _strip_conditional_prefix(
"If she works 10 hours every day for 5 days, "
"how much money does she make?"
)
assert out == "How much money does she make?"
def test_uppercases_leading_letter(self) -> None:
out = _strip_conditional_prefix(
"If Jen has 150 ducks, how many total birds does she have?"
)
assert out is not None and out.startswith("How ")
def test_returns_none_when_no_prefix(self) -> None:
assert _strip_conditional_prefix("How many oysters in 2 hours?") is None
def test_returns_none_on_malformed_if(self) -> None:
assert _strip_conditional_prefix("If") is None
def test_handles_lowercase_if(self) -> None:
# Case-insensitive prefix match.
out = _strip_conditional_prefix(
"if Marnie has 50 beads, how many bracelets can she make?"
)
assert out is not None
assert out.startswith("How many bracelets")
def test_is_pure_deterministic(self) -> None:
prompt = "If x, how many y does z have?"
first = _strip_conditional_prefix(prompt)
second = _strip_conditional_prefix(prompt)
assert first == second
def test_no_mutation_of_input(self) -> None:
prompt = "If z, how many y does w have?"
original = str(prompt)
_strip_conditional_prefix(prompt)
assert prompt == original # str immutability check + no in-place edit
# ---------------------------------------------------------------------------
# _filtered_question_choices recovers via prefix strip when the bare
# parser fails
# ---------------------------------------------------------------------------
class TestQuestionRecovery:
def test_bare_how_many_still_admits(self) -> None:
"""Pre-D.3 path stays — bare ``How many X does Y have?`` admits
without conditional-prefix recovery."""
out = _filtered_question_choices("How many apples does Tom have?")
# The parser handles this; recovery should not fire.
assert out, "bare canonical form must still admit"
def test_conditional_prefix_recovers(self) -> None:
"""ADR-0163.D.3 — ``If X, how many Y does Z have?`` admits via
prefix-strip recovery."""
with_prefix = "If something, how many apples does Tom have?"
bare = "How many apples does Tom have?"
recovered = _filtered_question_choices(with_prefix)
baseline = _filtered_question_choices(bare)
assert len(recovered) == len(baseline)
def test_non_question_returns_empty(self) -> None:
assert _filtered_question_choices("Tom has 5 apples.") == []
def test_recovery_returns_empty_when_suffix_unparseable(self) -> None:
"""If the suffix is still outside the parser's accepted shapes,
admission stays empty. No false admission. wrong=0 by
construction."""
out = _filtered_question_choices(
"If X, this is not even a question shape."
)
assert out == []
# ---------------------------------------------------------------------------
# Real GSM8K train_sample examples — the cases this phase is designed
# to lift. These are the literal questions from the still-refused 38
# that carry the conditional-prefix shape.
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"question",
[
"If she works 10 hours every day for 5 days, how much money does she make?",
"If 50 beads are used to make one bracelet, how many bracelets will Marnie be able to make out of the beads she bought?",
"If Jen has 150 ducks, how many total birds does she have?",
],
)
def test_real_gsm8k_conditional_prefixes_strip_cleanly(question: str) -> None:
"""The prefix-strip operation succeeds on real GSM8K conditional
questions. Whether the SUFFIX is admissible by the parser is a
separate concern this test pins only the deterministic prefix
removal.
"""
stripped = _strip_conditional_prefix(question)
assert stripped is not None
assert stripped.lower().startswith("how ")