feat(adr-0163-f2): EX-6 hyphen-bonded unit extraction — pseudo-accumulation confusers refuse (wrong 7->5) (#473)

The confuser probe's two pseudo-accumulation misfires (0005 ->796, 0007 ->996)
both traced to the same extraction blind spot: a number bonded to its unit by a
hyphen (`25-foot sections`, `20-inch pieces`) was invisible to the base
`number + space + word` pattern, so the self-verification completeness clause
never saw the divisor and the bare `buys ... gives` accumulation read as
"complete". This is the highest-leverage lever the 2026-05-29 session named
("the gate must see the fractions/25-foot it currently misses").

EX-6 adds a tight, ADR-0165-safe lexeme pass: a digit run, a single hyphen, an
alphabetic unit word. The alphabetic-only unit group keeps numeric ranges (`3-5`)
out; taking only the first hyphen segment keeps the postmodifier tail
(`25-year-old`) from inflating the unit — so it stays clear of the deferred EX-3
multi-word-unit traps. Over-extraction here is strictly refuse-preferring: making
the divisor visible drives 0005/0007 to refuse via the polarity-None
`cuts`/`splits` clause, never to a wrong answer.

Evidence (deterministic, the microscope):
- confuser probe: wrong 7 -> 5; pseudo-accumulation 0 wrong / 4 refused;
  genuine positives still 7 solved; pair-tells unchanged (4).
- train_sample (capability): 3/47/0 byte-identical.
- practice accumulation: 3/47/0 (wrong=0) byte-identical.
- smoke 67 passed; lane-SHA freeze 8/8 (serving frozen).

Tests:
- TestEX6HyphenatedUnitNumbers pins the new lexeme (value+unit, decimal, no
  double-count, word-compound unaffected, numeric range not read as a unit).
- TestProbeBaseline tightened wrong 7->5; new test_pseudo_accumulation_does_not_misfire
  is the failing-under-violation obligation (fails loudly if the pass regresses).
- TestSlashFractionLeakHazard pins the deferred `1/4`->`4` denominator leak: not
  fixed here because suppressing the leaked operand *removes* a quantity and can
  unblock the completeness clause (not unambiguously refuse-preferring), so it
  needs its own train_sample + probe validation.
This commit is contained in:
Shay 2026-05-29 12:23:26 -07:00 committed by GitHub
parent 62d9db7a7c
commit 475d44d245
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 121 additions and 6 deletions

View file

@ -30,6 +30,13 @@ this module, so none of this can move the serving ``3/47/0``):
units are blanked (empty, like a sentence-final number): the value stays
grounded, the unit is honestly unknown. Closed lexeme set (``_NON_UNIT_WORDS``),
not a grammar template (ADR-0165).
* **EX-6 hyphen-bonded number-units** (ADR-0163-F2). A number bonded to its unit
by a hyphen (``25-foot``, ``20-inch``, ``2.5-mile``) was invisible to the base
``number + space + word`` pattern, so the completeness check never saw the
divisor the blind spot behind the pseudo-accumulation confusers (0005/0007).
Tight lexeme (digit run, hyphen, alphabetic unit word); the alphabetic-only unit
group keeps numeric ranges (``3-5``) out and only the first hyphen segment is
taken, so it stays clear of the deferred EX-3 multi-word-unit traps below.
EX-3 (multi-word units) is deliberately **not** integrated. Two distinct traps
defeat the tightest lookahead-anchored rule the brief admits:
@ -94,6 +101,19 @@ _FINAL_NUMBER_RE: Final[re.Pattern[str]] = re.compile(
r"(?<![\w.])(\d+(?:\.\d+)?)(?=\s*(?:[.?!]|$))"
)
# EX-6: a number bonded to its unit by a hyphen (``25-foot``, ``20-inch``,
# ``2.5-mile``). The base ``_QTY_RE`` requires whitespace before the unit word,
# so a hyphen-bonded unit was invisible — the blind spot behind the
# pseudo-accumulation confusers (``25-foot`` / ``20-inch`` divisors the
# completeness check never saw). Tight lexeme: a digit run, a single hyphen, an
# alphabetic unit word. The trailing ``[a-zA-Z]+`` (not ``\d``) keeps numeric
# ranges (``3-5``) out, and taking only the first hyphen segment keeps the
# postmodifier tail (``25-year-old`` -> unit ``year``) from inflating the unit —
# so this stays clear of the deferred EX-3 multi-word-unit traps.
_HYPHEN_QTY_RE: Final[re.Pattern[str]] = re.compile(
r"(?<![\w.])(\d+(?:\.\d+)?)-([a-zA-Z]+)"
)
# Function words that are never units. When the token immediately after a number
# is one of these (``$0.75 each``, ``$40 to go``, ``3/4 of``), the single-word unit
@ -162,9 +182,10 @@ def extract_quantities(problem_text: str) -> tuple[Quantity, ...]:
passes never double-count a number:
1. EX-4 same-unit list (claims every number in the list);
2. digit + single unit word (skips numbers a list already claimed);
3. EX-1 word-number + unit word (alphabetic, disjoint from digit spans);
4. EX-5 sentence-final bare number (skips any already-claimed digit).
2. EX-6 hyphen-bonded number-unit (``25-foot``; claims the digit span);
3. digit + single unit word (skips numbers a list/hyphen pass already claimed);
4. EX-1 word-number + unit word (alphabetic, disjoint from digit spans);
5. EX-5 sentence-final bare number (skips any already-claimed digit).
"""
found: list[tuple[int, Quantity]] = []
claimed: list[tuple[int, int]] = []
@ -179,6 +200,16 @@ def extract_quantities(problem_text: str) -> tuple[Quantity, ...]:
found.append((pos, quantity))
claimed.append((pos, pos + len(num.group(0))))
# 1b. EX-6 — hyphen-bonded number-unit (``25-foot``). Claims the digit span so
# the bare/final passes never re-surface the number with a blank unit.
for match in _HYPHEN_QTY_RE.finditer(problem_text):
if _claimed(match.start(1), claimed):
continue
quantity = _quantity(match.group(1), match.group(2))
if quantity is not None:
found.append((match.start(1), quantity))
claimed.append(match.span(1))
# 2. digit + single unit word — the original base pattern.
for match in _QTY_RE.finditer(problem_text):
if _claimed(match.start(1), claimed):

View file

@ -17,9 +17,14 @@ from evals.gsm8k_math.confusers.v1.runner import (
summarize,
)
# Honest measured baseline (2026-05-29). The probe revealed the sealed composers
# wrongly answer these confuser categories; this pins them so they cannot grow.
_BASELINE_WRONG = 7
# Honest measured baseline. The probe revealed the sealed composers wrongly answer
# these confuser categories; this pins them so they cannot grow.
# 2026-05-29 (initial): wrong=7, pair_tells=4.
# 2026-05-29 (EX-6 hyphen-bonded units, ADR-0163-F2): the two pseudo-accumulation
# misfires (0005->796, 0007->996) now refuse — the 25-foot/20-inch divisor is
# finally visible to the completeness/polarity gate. wrong 7->5. pair_tells
# unchanged (the pseudo-accumulation cases carry no positive twin).
_BASELINE_WRONG = 5
_BASELINE_PAIR_TELLS = 4
@ -68,5 +73,19 @@ class TestProbeBaseline:
def test_pair_tells_do_not_regress(self) -> None:
assert len(pair_inconsistencies()) <= _BASELINE_PAIR_TELLS
def test_pseudo_accumulation_does_not_misfire(self) -> None:
# ADR-0163-F2 EX-6: the hyphen-bonded-unit extraction made the
# 25-foot/20-inch divisor visible, so the pseudo-accumulation confusers
# (0005, 0007) refuse instead of committing 796/996. This is the specific
# obligation the fix proves — it fails loudly if the hyphen pass regresses
# (the gate would again miss the divisor and the category would misfire).
results = run_probe()
pseudo = [r for r in results if r.category == "pseudo-accumulation"]
assert pseudo, "pseudo-accumulation cases missing from corpus"
assert all(r.verdict != "wrong" for r in pseudo), (
"pseudo-accumulation confuser misfired — the hyphen-bonded-unit "
"divisor is no longer reaching the completeness/polarity gate."
)
def test_deterministic(self) -> None:
assert summarize(run_probe()) == summarize(run_probe())

View file

@ -94,6 +94,71 @@ class TestNoRegression:
assert _triples("It costs 0.75 dollars.") == [(0.75, "dollars", "0.75")]
class TestEX6HyphenatedUnitNumbers:
"""ADR-0163-F2 — hyphen-bonded number-unit tokens (``25-foot``, ``20-inch``).
The base ``_QTY_RE`` requires ``number + whitespace + unit word``, so a number
bonded to its unit by a hyphen (``25-foot sections``, ``20-inch pieces``) was
invisible the very blind spot behind the pseudo-accumulation confusers
(0005 ``796``, 0007 ``996``): the completeness clause could not see the
``25``/``20`` divisor, so the accumulation reading was wrongly "complete".
This is a tight, ADR-0165-safe lexeme pattern (digit run, a hyphen, an
alphabetic unit word) strictly distinct from the deferred EX-3 multi-word
*space-separated* unit problem. Over-extraction here only costs refusals
(the gate is refuse-preferring), never a wrong answer.
"""
def test_hyphen_bonded_unit_extracts_value_and_unit(self) -> None:
assert _triples("She cuts it into 20-inch pieces.") == [(20.0, "inch", "20")]
def test_hyphen_bonded_unit_mid_sentence(self) -> None:
assert _triples("She splits it into 25-foot sections.") == [(25.0, "foot", "25")]
def test_decimal_hyphen_bonded_unit(self) -> None:
assert _triples("He ran a 2.5-mile loop.") == [(2.5, "mile", "2.5")]
def test_hyphen_unit_not_double_counted_as_final_number(self) -> None:
# the hyphen pass claims the digit span; the bare-final pass must not
# also surface "25" with an empty unit.
assert len(extract_quantities("It was 25-foot.")) == 1
def test_word_compound_still_takes_word_number_path(self) -> None:
# digit-hyphen is a different lexeme from the EX-1 word-word compound;
# "twenty-four" must still resolve via the word-number path, unaffected.
assert _triples("The team scored twenty-four points.") == [
(24.0, "points", "twenty-four"),
]
def test_numeric_range_is_not_read_as_unit(self) -> None:
# "3-5" is digit-hyphen-DIGIT; the unit group requires letters, so the
# hyphen pass must not fire and invent a unit from the second number.
assert all(q.unit != "5" for q in extract_quantities("Pick 3-5 apples."))
class TestSlashFractionLeakHazard:
"""Honest pin (deferred hazard): a slash-fraction leaks its denominator.
``"gives 1/4 to a friend"`` extracts the bare ``4`` (the base ``_QTY_RE``
sees ``4`` + space + the function-word ``to``, blanks the unit, and emits a
standalone quantity ``4``). This is a grounded-but-wrong operand it is the
second half of the pseudo-accumulation misfire (``1000 - 4 = 996``).
It is **not** fixed in this PR on purpose. Suppressing the leaked ``4`` *removes*
a quantity, which can *unblock* the completeness clause (a derivation that was
"incomplete" because it ignored the spurious ``4`` could become "complete" and
commit) i.e. the fix is not unambiguously refuse-preferring and needs its own
train_sample + probe validation. The hyphen-unit pass (TestEX6) already drives
confusers 0005/0007 to *refuse* via the polarity-None ``cuts``/``splits`` clause,
so this leak is currently dormant behind that refusal. This test pins the leak
so a future fraction-operand PR addresses it deliberately, not by accident.
"""
def test_slash_fraction_denominator_currently_leaks(self) -> None:
# documents current (leaky) behavior — flip this when fractions are modeled.
assert (4.0, "", "4") in _triples("She gives 1/4 to a friend.")
class TestEX3StillDeferred:
"""Honest pin: EX-3 (multi-word units) remains deferred after the Track C redo.